CeNTIE is supported by the Australian Government through the Advanced Networks Program (ANP) of the Department of Communications, Information Technology

CeNTIE is supported by the Australian Government through the Advanced Networks Program (ANP) of the Department of Communications, Information Technology and the Arts

The Continuous Media Web, Annodex™, CMML, and Aida are being developed by the CSIRO ICT Centre in the CeNTIE project

Full integration of Audio and Video into the World Wide Web

The Continuous Media Web

Silvia Pfeiffer, CSIRO ICT Centre, [email protected]

24th July 2005

Convergence between diverse platforms for the distribution of digital media Internet Protocol;e.g. 3G mobile phones, IP-TV

Integration of meta data along the digital content lifecycle

Web is the standard platform for identification and delivery of services

Google is the standard means of searching and finding anything

Challenges for Digital Media

The idea behind the CMWeb

Searching media using google

Direct addressing of points of interest

Hyperlinking from inside media to other Web resources

Integrated handling of metadata with the content

Dynamic composing of media web content

Consequence:Development of a new standard through the IETF

Searching: The Video Web Search Engine

The Panoptic search engine allows searching for video clips using key words.

A hyperlink in the result list allows direct access to the clip of interest.

The search engine use metadata and free text annotations to index the files and retrieve the result list.

Surfing: The Web Browser extension

A Firefox Web Browser extension allows streaming of videos. The components are:

• A clip table to allow direct access to the sections of the video.

• Free text annotation per video clip.

• A hyperlink per video clip.

• A data display section (for the video itself).

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

The Aida Mobile Phone Browser



Searching for clips



Hyperlinking between videos,Browsing history



Clip table

The Technical Components

CMML

CMMLSearch Engine

Annodex

Annodex

Annodex

Annodex

Browser

WebServer

URI

URI URI

URI

Future IP-based Consumer Media



The Technical Components

CMML: Continuous Media Markup-Language, the HTML of time-continuous data, which allows to markup media

Annodex: Annotated and Indexed Time-Continuous Data Container, which allows synchronised and interleaved exchange of CMML and multi-track binary data over networks

Hyperlinks into Annodex/CMML: temporal offsets & named clips, which allows direct addressing of information inside media using URLs

Enriching Video with CMML

<cmml>

<stream basetime=“0” utc=“20040114T153500.00Z”>

<import src=“galaxies.mpg” contenttype=“video/mpeg" start=“npt:0"/>

</stream>

<head>

<title>Hidden Galaxies</title>

<meta name="author" content="CSIRO"/>

</head>

<clip id=“findingGalaxies” start=“15”>

<a href=“http://www.aao.gov.au/galaxies.anx#radio”>Related video on Detection of Galaxies</a>

<img src=“galaxy.jpg”/>

<desc>What’s out there? </desc>

<meta name=“KEYWORDS" content=“Radio Telescope, Galaxies"/>

</clip>

</cmml>

CMML: i18n

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><!DOCTYPE cmml SYSTEM "cmml.dtd"><cmml lang="pl">

<head lang="en"> <title>Types of fish</title> <meta name="Producer" content="Joe Ordinary"/> <meta name="DC.Author" content="Joe's friend"/></head>

<clip lang="de" id="intro" start="npt:0"> <a href="http://www.blah.au/fish.html">Lesen Sie mehr über Fische.</a> <desc lang="en">This is the introduction to the film Joe made about fish.</desc></clip>

<clip lang="en" id="goldfish" start="npt:5:5.9"> <a href="http://www.blah.au/morefish.anx?id=goldfish">More video clips on goldfish.</a> <img src="http://www.blah.au/goldfish.jpg"/> <desc>Joe has a fishtank at home with many colourful fish.</desc> <meta name="Subject" content="goldfish"/> <meta lang="fr" name="sujet" content="poisson d'or"/></clip></cmml>

CMML: multiple annotation tracks

“track” attribute of clips allows grouping intomultiple annotation tracks:

<clip track="authors" start="npt:0"> <a href="http://www.blah.au/joe.html">Go to Joe’s home page.</a> <desc>Joe recorded and edited this part of the film.</desc></clip>

<clip track="default" start="npt:0"> <a href="http://www.blah.au/fish.html">Read more about fish.</a> <desc>This is the introduction to the film Joe made about fish.</desc></clip>

<clip track=”authors" start="npt:3.5"> <a href=”http://www.blah.au/claudine.html”>Got to Claudine’s home page.</a> <desc>Claudine recorded this bit of the film.</desc> <meta name="Author" content="Claudine"/></clip>

Authoring Annodex from CMML and media

CMML file

Annodex file

continuous media file

Merger

Conceptual Annodex File

media mediaanchor

Annodexed mediaactivehead anchor

head

<clip id=“findingGalaxies"> <a href=“http://www.aao.gov.au/> Related video on Detection of Galaxies"</a>

<img src=”galaxy.jpg”/>


<meta name=“KEYWORDS" content=“Astronomy,Radio Telescope, Galaxies"/>

</a>

<clip id=“findingGalaxies"> <a href=“http://www.aao.gov.au/> Related video on Detection of Galaxies"</a>

<img src=”galaxy.jpg”/>


<meta name=“KEYWORDS" content=“Astronomy,Radio Telescope, Galaxies"/>

</a>

<head>



</head>

<head>



</head>

galaxies.mpg

start=15timebase=0

A multi-track Annodex file

CMML file

Annodex file

continuous media files

Merger(anxenc)

...

URL Addressing Schemes

Addressing of time offset / segment:http://www.foo.bar/csiro.anx?t=smpte-25:00:01:25:07

http://www.foo.bar/csiro.anx?t=npt:85.28-105

http://www.foo.bar/csiro.anx?t=clock:20030327T103045.25Z

Addressing of named clips:

http://www.foo.bar/csiro.anx?id=”findingGalaxies”

http://www.foo.bar/csiro.anx#findingGalaxies

Temporal URI format

Other sites and users can link directly to a point of interest in a media resource.

A common format for this allows tools to generate the correct URI to retrieve the given playback time, without the author of the referenced content explicitly naming it.

The point of interest can be linked to from web pages, email and other media resources.

What software exists?

ContentOwner

ContentProvider

Annodexing Serving &Searching

Annodex Authoring

NetworkProvider

AnnodexServer:

Apache

Search

Delivery

CSIROContinuousMedia WebTechnology

Consumer





MobileDelivery

Browsers

How to create Annodex content?

A Windows authoring software exists (the “FrontPage” of the Continuous Media Web).

It allows simple authoring of clips at the press of one button with automatic extraction of a representative image.

Entry of textual annotations in the form of free text (descriptions) and structured text is supported.

Also entry of hyperlinks (anchors) is provided.

Dynamic content creation/syndication from collections of clips, e.g. news sites, video blogs

ANXCGI

Metadata DB

(incl. references to media clips)

Media Clip Collection

Research: Scripting of media content

Use signal analysis for automating the CMML creation task, e.g. speaker segmentation for meeting recordings + speech recognition

CMML DB

Video/Audio Analysis

Research: Automated Annodexing

Use XSLT for transcoding between different XML languages, e.g. NewsML to CMML

Database backend applications

Metadata DBCMML DB

Transform

Research: Transcoding of meta data

The CSIROwww.annodex.net

Standards for Enriched Media

SMIL = Synchronised Multimedia Interaction Language

MPEG-7 = open framework for describing Multimedia content

MPEG-21 = open framework for multimedia delivery and consumption

CMWeb = open standards for annotating and indexing Web media (hypermedia)

Documents

CeNTIE is supported by the Australian Government through the Advanced Networks Program (ANP) of the Department of Communications, Information Technology