PC Chairs Lynda Hardman Alex Hauptmann Nadia Magnenat-Thalman

<is web> Information Systems & Semantic WebUniversity of Koblenz ▪ Landau, Germany

General ChairsDietrich PaulusSteffen Staab

SAMT 2008 Semantics and Digital Media Technology

Koblenz, December 3-5, 2008http://samt2008.uni-koblenz.de

PC ChairsLynda HardmanAlex Hauptmann

Nadia Magnenat-Thalman


Semantic Web as a Knowledge Infrastructure for Multimedia

Steffen Staab

<is web>

ISWeb - Information Systems & Semantic Web

Steffen [email protected]

MMKM3 of 28

Depict ISWeb People!

Context is missing!

Precision

12/12

Recall

12/19 (FTE)

12/33 (all)

<is web>



MMKM4 of 28


There is a picture of ISWeb in 2006 in the Wiki!

<is web>



MMKM5 of 28


Some more ISWeb members you find on our Web page!

<is web>



MMKM6 of 28


Outside of our Web Site you may also discover current members…

<is web>



MMKM7 of 28


For others, you need to integrate information from the Web and from Word documents of my mailbox

<is web>



MMKM8 of 28

Knowledge Infrastructure

A Multimedia objects exist in an environment.

This environment (context) can and should be used to find, access and present the objects.

Knowledge about Multimedia Objects exists in an environment, too.

This knowledge infrastructure can and should be used to find, access and present the objects.

Well, known in the multimedia field

Little explored until now

<is web>



MMKM9 of 28

Structure of the Environment

Knowledge in Portal

Knowledge in File System

Knowledge in Wiki

Knowledge in Outside Service

Knowledge in Mailbox

<is web>



MMKM10 of 28

Structure of the Environment1. Collect or Access

2. Harvest/Query Knowledge

3. Collect & Organize Knowledge

4. Use Knowledge

<is web>



MMKM11 of 28

What does the Knowledge Infrastructure need?

1. Ontologies Multimedia Ontologies Context Ontologies Domain Ontologies

2. APIs

3. Querying and reasoning

<is web>



MMKM12 of 28

How to structure this knowledge?

PersonRegion

<is web>



MMKM13 of 28

Scenario

The "ISWeb" at Bad Kreuznach (team web page)

Localize a region (person)Annotate the content (interpretation)Carsten Saathoff, aceMedia Project lead, ISWeb, Semantic Multimedia, KAT Annotation ToolLink to knowledge on the Web:Reg1 foaf:depicts kodemaniak:mekodemaniak:me rdfs:label “Carsten Saathoff"kodemaniak:me rdf:type foaf:Person

Reg1

Summer School Semantic Web-05 (video)

http://en.wikipedia.org/wiki/Allies_of_World_War_II

http://www.reuters.com/news/video/summitVideo?videoId=56114

<is web>



MMKM14 of 28

Problem

The "ISWeb" at Bad Kreuznach

(team web page)Summer School Semantic Web-05 (video)

Seq1

Seq4

Multimedia objects are complex Compound information objects, fragment identification

Semantic annotation Subjective interpretation, context dependent

Linked data principle Open to reuse existing knowledge

Multimedia O

Domain O

Context O

Reg1

http://en.wikipedia.org/wiki/Allies_of_World_War_II

http://www.reuters.com/news/video/summitVideo?videoId=56114

<is web>



MMKM15 of 28

Reg1

Example: Fragment Identification with COMM

http://isweb.uni-koblenz.de/People/isweb-2007.jpg

core:image-data

http://en.wikipedia.org/wiki/Image:Yalta_Conference.jpg

loc:spatial-mask-role

5 25 10 20 15 15 10 10 5 15"^ x̂sd:string

dns:realized-by

loc:region-locator-descriptor

loc:bounding-box

dns:plays

dns:played-bydns:defines

data:has-rectangle


<is web>



MMKM16 of 28

Example: Region Annotation with COMM

core:image-data

http://en.wikipedia.org/wiki/Image:Yalta_Conference.jpg

loc:spatial-mask-role

5 25 10 20 15 15 10 10 5 15"^^xsd:string

dns:realized-by

loc:region-locator-descriptor

loc:bounding-box

dns:plays

dns:played-bydns:defines

data:has-rectangle

dns:setting core:semantic-annotation

core:semantic-label-role

dns:defines

dns:played-by

http://en.wikipedia.org/wiki/Churchill

foaf:Person

rdf:type

Reg1


http://kodemaniak.de/foaf.rdf#me

What is important here, is what you don‘t see!

<is web>



MMKM17 of 28

Noam

Steffen“I apply for this position. Here

is my cover letter and CV ...”

Bercovici.pdf

Mrs. Werger

Bercovici.pdf

“Please prepare papers for his hire,

Mrs. Werger“

“Thanks for your application and here is some information

about the NeOn project…” Colleagues

Another Scenario ...

<is web>



MMKM18 of 28

… Contextualized Information

Bercovici.pdfNoam

“I apply for this position. Here is my cover letter and CV ...”

Colleagues

Context of multimedia information is lost

<is web>



MMKM19 of 28

… Communicating

VoIP

Steffen

Noam

Communication Context of multimedia information is lost

<is web>



MMKM20 of 28

…classifying information…

Classification of multimedia information is lost

<is web>



MMKM21 of 28

Descriptions and Situations

Descri

ption

s and

Situ

ation

sDescriptions and Situations

Descri

ption

s and

Situ

ation

s

X-COSIM Ontology

Sharing, Arguing, Explaining

Filing, Downloading, Modifying,Converting

Structuring,Summarizing,User-defined Ontologies

Creating, Reusing, Copy & Paste

<is web>



MMKM22 of 28

DOLCE for Developers of E-mail Clients?

Particular

EndurantPerdurantAbstract Quality

DescriptionSituation

X-COSIMA

SPARQLRDF API

RDF Store

X-COSIMO

File Manager, Browser, Mail Client, Task Manager, Text Processor, ...

Dublin Core, Calendar, Foaf, ...

ReferenceModel,Storage

Apps,DomainModels

DataAccess,Mapping

Map between contextualized representations and the reference model

X-COSIM Architecture

<is web>



MMKM23 of 28

Resulting Meta Data

ClassificationAspect

RealizationAspects

CommunicationAspect

Noam

Steffen

Noam

<is web>



MMKM24 of 28

What does the Knowledge Infrastructure need?

1. Ontologies Multimedia Ontologies Context Ontologies Domain Ontologies

2. APIs

3. Querying and reasoning

Jump to end

<is web>



MMKM25 of 28

Where are these metadata?

Noam

Steffen

Nasir

Email

MMDB

Server

<is web>



MMKM26 of 28

Federation of Knowledge

Nasir

<is web>



MMKM27 of 28

:NasirFOAF{:NasirFOAF dc:creator :Nasir .:Nasir :depictIn http://nas.jpg.}

:ISWebWiki{kodemaniak:me :member :ISWeb .Kodemaniak:me :depictIn :WikiImg1 .:ISWeb :depictIn http://..wiki1.jpg .}

Networked Graphs

:ISWebFOAF g:definedBy „CONSTRUCT {?X :depictIn...“ . }

:ISWebFOAF{:ISWeb foaf:name „Info Syst…“.:ISWeb :member :Nasir.:ISWeb :depictIn http://..wiki1.jpg.

Images of team members depict part

of the group

:ISWeb :depictIn http://nas.jpg .

Copy images that depict ISWeb

<is web>



MMKM28 of 28

:NasirFOAF{:NasirFOAF dc:creator :Nasir .:Nasir :depictIn http://nas.jpg.}

:ISWebWiki{kodemaniak:me :member :ISWeb .Kodemaniak:me :depictIn :WikiImg1 .:ISWeb :depictIn http://..wiki1.jpg .}

Meta Knowledge

:ISWeb :depictIn http://nas.jpg .

:ISWeb :depictIn http://..wiki1.jpg.

:ISWebFOAF{:ISWeb foaf:name „Info Syst…“.:ISWeb :member :Nasir.

:ISWebFOAFMeta{:ISWebFOAF provenance :NasirFOAF .:ISWebFOAF provenance

:ISWebWiki . }

<is web>



MMKM29 of 28

Take home message

Knowledge Structures of Multimedia Knowledge Infrastructure Knowledge CaptureSemantic Web ties them together

Some first steps are done: Standardization of Semantic Weblanguages RDF carried along with some media:

RDF in PDF, RDF-A in Open Document Format

Interesting Issues: Standardization of generic protocols Management of heterogeneity

of data, applications and services Mixed systems of reasoning and low level extraction ….

<is web>



MMKM30 of 28

Systems & Literature

COMM Ontology & API: http://comm.semanticweb.org/

X-Cosim Ontology & API:http://isweb.uni-koblenz.de/Research/x-cosim

Networked graphs: http://isweb.uni-koblenz.de/Research/NetworkedGraphs

Querying for meta knowledge:http://isweb.uni-koblenz.de/Research/MetaKnowledge

Distributed querying:http://isweb.uni-koblenz.de/Research/DistributedSPARQL

ISWC-2007

K-Cap-2007

WWW-2008a

WWW-2008b

…to be written…

http://comm.semanticweb.org/

http://isweb.uni-koblenz.de/Research/x-cosim

http://isweb.uni-koblenz.de/Research/NetworkedGraphs

http://isweb.uni-koblenz.de/Research/MetaKnowledge

http://isweb.uni-koblenz.de/Research/DistributedSPARQL


Thank You!


General ChairsDietrich PaulusSteffen Staab

SAMT 2008 Semantics and Digital Media Technology

Koblenz, December 3-5, 2008http://samt2008.uni-koblenz.de

PC ChairsLynda HardmanAlex Hauptmann

Nadia Magnenat-Thalman

Documents

PC Chairs Lynda Hardman Alex Hauptmann Nadia Magnenat-Thalman