Embed Size (px)
DESCRIPTION
The semantics of rich multimedia presentations in the web such as SMIL, SVG, and Flash cannot or only to a very limited extend be understood by search engines today. This hampers the retrieval of such presentations and makes their archival and management a difficult task. Existing metadata models and metadata standards are either conceptually too narrow, focus on a specific media type only, cannot be used and combined together, or are not practically applicable for the semantic description of rich multimedia presentations. In this paper, we propose the Multimedia Metadata Ontology (M3O) for annotating rich, structured multimedia presentations. The M3O provides a generic modeling framework for representing sophisticated multimedia metadata. It allows for integrating the features provided by the existing metadata models and metadata standards. Our approach bases on Semantic Web technologies and can be easily integrated with multimedia formats such as the W3C standards SMIL and SVG. With the M3O, we unlock the semantics of rich multimedia presentations in the web by making the semantics machine-readable and machine-understandable. The M3O is used with our SemanticMM4U framework for the multi-channel generation of semantically-rich multimedia presentations.
Citation preview
Web Science & Technologies
University of Koblenz ▪ Landau, Germany
Unlocking the Semantics of Multimedia Presentations in the Web with the Multimedia Metadata Ontology
Carsten Saathoff and Ansgar Scherp
Carsten Saathoff <[email protected]> 4 of 18WWW 2010, Raleigh, NC, USA
How to …
…index multimedia presentations? …search and navigate among collections
of presentations? …enable semantic access?
Metadata!
Carsten Saathoff <[email protected]> 5 of 18WWW 2010, Raleigh, NC, USA
Existing Standards
Focus on single media typeNo support for “true” multimedia
No interoperabilityMetadata distributed over numerous formats
No support for semanticsNo integration with background knowledge
Carsten Saathoff <[email protected]> 6 of 18WWW 2010, Raleigh, NC, USA
Embedding Metadata (Example: SMIL)<smil xmlns="http://www.w3.org/2006/SMIL30/..."><head>
<!-- Metadata --><metadata id="meta-rdf">
</metadata><!-- layout --></head><body> <!-- Presentation content --> <seq id="mainPresentation"> <par xml:id="scientificAchievements">...</par> <par xml:id="scientificRisks">...</par> </seq></body></smil>
???
Carsten Saathoff <[email protected]> 7 of 18WWW 2010, Raleigh, NC, USA
Solution
Generic and formal modeling framework for representing
multimedia metadata.
M3O: Multimedia Metadata Ontology
Carsten Saathoff <[email protected]> 8 of 18WWW 2010, Raleigh, NC, USA
M3O at a glance
Covers core features of existing standards
Based on Semantic Web standardsRDF for data representation
Realized as a core ontologyRepresented in OWL
Carsten Saathoff <[email protected]> 9 of 18WWW 2010, Raleigh, NC, USA
Supported Features
Information Realization
Annotation Decomposition
Provenance Identification of Resources
Achievements…
Albert Einstein
vs.
Carsten Saathoff <[email protected]> 10 of 18WWW 2010, Raleigh, NC, USA
Semantic Web – RDF
ex:presentation-nuclear
ex:depicts
dbpedia:Albert_Einstein
“Albert Einstein”
foaf:name dbpedia:Ulm
dbpedia:birthplace
dbpedia:Germany
foaf:Person
rdf:type
dbpedia:locatedIn
dbpedia:City
dbpedia:Country
rdf:type
rdf:type
ex:Scientist
Carsten Saathoff <[email protected]> 11 of 18WWW 2010, Raleigh, NC, USA
Core Ontologies
Based on ontology design patterns Aligned to upper level ontology
Provide Rich Axiomatization Modularity Extensibility Reusability Separation of Concerns
Successfully applied to other domains Events (F), PIM (X-COSIMO), MPEG-7 (COMM)
Carsten Saathoff <[email protected]> 12 of 18WWW 2010, Raleigh, NC, USA
Naïve Approach to Annotation
dbpedia:Albert_Einsteinex:presentation-nuclear
m3o:depicts
ex:john
m3o:creator
Reification
Carsten Saathoff <[email protected]> 13 of 18WWW 2010, Raleigh, NC, USA
Annotation in M3O
ex:annotation-situation
ex:annotation-pattern
ex:annotated-concept ex:annotation-concept
dbpedia:Albert_Einsteinex:presentation-nuclear
defines
hasSetting
classifies classifiessatisfies
Description
Situation
Carsten Saathoff <[email protected]> 14 of 18WWW 2010, Raleigh, NC, USA
Provenance in M3O
ex:applied-method-role ex:creator
ex:manual-annotation ex:john
ex:annotation-situation
ex:annotation-pattern
ex:annotated-concept ex:annotation-concept
ex:presentation-nuclear
classifies classifies
satis
fies
dbpedia:Albert_Einstein
classifies classifies
defines
hasSetting
Carsten Saathoff <[email protected]> 16 of 18WWW 2010, Raleigh, NC, USA
M3O Patterns
Information Realization
Annotation
Data Value
Decomposition
Carsten Saathoff <[email protected]> 17 of 18WWW 2010, Raleigh, NC, USA
Embedding M3O in SMIL<metadata id="meta-rdf"><rdf:RDF xml:base="http://example.com/john/nuclear" xmlns:dul="http://www.loa-cnr.it/ontologies/DUL.owl#" xmlns:m3o="http://m3o.semantic-multimedia.org/ontology/model.owl#"> <m3osmil:SMILFile rdf:about="http://example.com/john/nuclear.smil"> <dul:realizes rdf:resource="#presentation-1"/></m3osmil:SMILFile><m3osmil:SMILElement
rdf:about="http://example.com/john/nuclear.smil#scientificAchievements"> <dul:realizes rdf:resource="#part-1"/></m3osmil:SMILElement><!-- more, e.g. #part-1, #image-1, ... --></rdf:RDF></metadata>
<body> <!-- Presentation content --> <seq id="mainPresentation"> <par xml:id="scientificAchievements">...</par> <par xml:id="scientificRisks">...</par> </seq></body></smil>
Carsten Saathoff <[email protected]> 18 of 18WWW 2010, Raleigh, NC, USA
Conclusions
M3O as guideline for modeling multimedia metadata SVG, SMIL, Flash foresee the use of Semantic
Web standards
Formalization provides for interoperability along media process chain
Integration with the Semantic Web http://semantic-multimedia.org/index.php/M3O:Main
Mapping to MPEG-7 via COMM Mapping to Media Ontology (W3C Media Annotations WG) planned Link to Java API
Carsten Saathoff <[email protected]> 20 of 18WWW 2010, Raleigh, NC, USA
Identified Features
Identification of Resources Information Realization Annotation Decomposition Provenance
Carsten Saathoff <[email protected]> 21 of 18WWW 2010, Raleigh, NC, USA
Features to Support
“Albert Einstein, Times Square”
creation-date:01/01/2010
Annotation
Annotation
Decomposition
Achievements…
Albert Einstein
“John”InformationRealization
Provenance
Carsten Saathoff <[email protected]> 22 of 18WWW 2010, Raleigh, NC, USA
M3O Patterns
D&S
Information Realization
Annotation
Data Value
Decomposition
Carsten Saathoff <[email protected]> 23 of 18WWW 2010, Raleigh, NC, USA
Information Realization
dul:realizes
dul:InformationRealization dul:InformationObject
rdf:type
rdf:type
Carsten Saathoff <[email protected]> 24 of 18WWW 2010, Raleigh, NC, USA
Information Realization Pattern
Carsten Saathoff <[email protected]> 25 of 18WWW 2010, Raleigh, NC, USA
Problems
Single MediaOnly few support multiple media types (e.g.
MPEG-7)None supports rich multimedia
presentations Limited Expressivity
No integration of “semantics” No guidelines for metadata use
Although SMIL, SVG, Flash foresee metadata and propose standards (RDF)
Carsten Saathoff <[email protected]> 26 of 18WWW 2010, Raleigh, NC, USA
Structure of a (SMIL) presentationImages
Achievements…
Text+ Video
Albert Einsteinspatial
temporal
sequential, parallel, …
+ Audio
Carsten Saathoff <[email protected]> 27 of 18WWW 2010, Raleigh, NC, USA
Carsten Saathoff <[email protected]> 28 of 18WWW 2010, Raleigh, NC, USA
Existing Standards
Most focus on single media type No support for “true” multimedia
No interoperability Metadata distributed over different standards
MPEG-7, ID3, Dublin Core, …
Carsten Saathoff <[email protected]> 29 of 18WWW 2010, Raleigh, NC, USA
Limited Expressivity
Integration of background knowledgeRefer to existing knowledge (LOD)
Semantics of Annotation Provenance of Metadata
Albert Einstein
Time Square
Nuclear Energy
Carsten Saathoff <[email protected]> 30 of 18WWW 2010, Raleigh, NC, USA
Naïve Approach to Annotation
Albert Einsteindepicts
“John”
creator
Reification
Anordnung wie auf naechster Folie
