Structure and Dynamics of Emergent Semantics Systemsnsfcac.rutgers.edu/conferences/icac/keynotes/icac_EmSem.pdfEmergent Semantics Systems Karl Aberer EPFL School of Computer and Communication

The National Centres of Competence in Research are aresearch instrument of the Swiss National Science Foundation

Structure and Dynamics of Emergent Semantics Systems

Karl Aberer

EPFL School of Computer and Communication Science

NCCR MICS, National Centre of Competence in Researchon Mobile Information and Communication Systems

[email protected]

http://www.mics.org/

http://lsirwww.epfl.ch/

2The National Centres of Competence in Research are aresearch instrument of the Swiss National Science Foundation

Overview

1. Emergent Semantics2. Mapping Inference in Semantic Overlay Networks3. Structure of Semantic Overlay Networks4. Peer Data Management Systems Implementation5. Outlook: Sensor Internet


Semantics

• Long-standing debate:”What is semantics?”• Standard response:

“Mapping of a syntactic structure into a semantic domain”

tiger

Syntactic structure: database, knowledge base

Semantic domain: real-world


Semantic Web

• Real-world is a somewhat ill-defined and hard to compute concept

• Proposal: Substitute real-world by shared formal conceptualization [Gruber 93]

tiger

Syntactic structure: database, knowledge base

Semantic domain: ontology

owl:tiger


The Issue with Ontologies

• What is the semantics of ontologies? After all, they are syntactic structures!

• Heterogeneous and evolving models and ontologies

tiger owl:tiger

rdf:tiger

oil:cat

wn:animal


The Correspondence Continuum

• Observation: Meaning is rarely a simple mapping from a syntactic structure to a semantic domain

• Continuum of (semantic) correspondences from symbol to (symbol to)* object [Smith 87]

• The meaning of a symbol is given by the composition of the semantic mappings that relate it to its root

• Instead of focusing on ever-richer modelling languages for concepts, focus on mapping languages and mapping discovery tools [Mylopolous 06]


Semantic Grounding

• Still: what should “relate ot the root mean”?• The meaning of symbols can be explained by its semantic

correspondences to other symbols alone [“Understanding understanding” Rapaport 93]

• Type 1 semantics: understanding in terms of something else– Problem: how to ground semantics?

• Type 2 semantics: understanding something in terms of itself– “syntactic semantics”: grounding through recursive understanding


Emergent Semantics

• Semantic correspondences form rich self-referential structures: recursive understanding

• Recursive understanding may converge against stable structures (fixpoints): emergent semantics [Aberer 04]

owl:tiger

rdf:tiger

oil:cat

wn:animalcontext


Peer-to-Peer Systems

• Resource Sharing (e.g. images)– No centralized infrastructure– Global scale information systems– Application-specific overlay networks


Efficiently Searching Resources (Data)

• Find images taken last week in Trondheim!

?


Resource Sharing

• What is shared?

content <rdf:Description about='' xmlns:xap='http://ns.adobe.com/xap/1.0/'>

<xap:CreateDate>2001-12-19T18:49:03Z</xap:CreateDate><xap:ModifyDate>2001-12-19T20:09:28Z</xap:ModifyDate><xap:Creator> John Doe </xap:Creator>

</rdf:Description>…

knowledge

bandwidth

storage

processing


Beyond Keyword Search

• Support of structured data at peers: schemas• Structured querying in peer-to-peer system

<xap:CreateDate>2001-12-19T18:49:03Z</xap:CreateDate><xap:ModifyDate>2001-12-19T20:09:28Z</xap:ModifyDate>

date?

<es:cDate> 05/08/2004 </es:cDate>

<myRDF:Date> Jan 1, 2005 </myRDF:Date>


Peer Data Management Systems

Q1=<GUID>$p/GUID</GUID> FOR $p IN /Photoshop_Image WHERE $p/Creator LIKE "%Robi%"

<Photoshop_Image><GUID>178A8CD8865</GUID><Creator>Robinson</Creator><Subject><Bag><Item>Tunbridge Wells

</Item><Item>Royal Council</Item>

</Bag> </Subject>…

</Photoshop_Image>

Photoshop(own schema)

<WinFSImage><GUID>178A8CD8866</GUID><Author> <DisplayName> Henry Peach Robinson

<DisplayName> <Role>Photographer</Role>

<Author><Keyword>Tunbridge

</Keyword><Keyword>Council</Keyword>…

</WinFSImage>

WinFS (known schema)

T12 =<Photoshop_Image><GUID>$fs/GUID</GUID> <Creator>$fs/Author/DisplayName

</Creator></Photoshop_Image>FOR $fs IN /WinFSImage

Q2=<GUID>$p/GUID</GUID> FOR $p IN T12WHERE $p/Creator LIKE "%Robi%"

⇒ Extending data integration techniques to decentralized settings


• Pairwise mappings– Local mappings overcome global heterogeneity– Iterative query reformulation

Semantic Heterogeneity in PDMS

<xap:CreateDate>2001-12-19T18:49:03Z</xap:CreateDate><xap:ModifyDate>2001-12-19T20:09:28Z</xap:ModifyDate>

date?

<es:cDate> 05/08/2004 </es:cDate>

<myRDF:Date> Jan 1, 2005 </myRDF:Date>

es:cDate xap:CreateDate

es:cDate

myRDF:Date myRDF:

Date

xap:

Modify

Date


PDMS vs. Classical Data Integration

• Traditional database techniques (e.g., LAV/GAV) rely on centralized schemas to integrate data sources

• Not applicable to large-scale, decentralized contexts– Scale: 100s vs. 10^3-10^6– Churn: no fixed topology– Autonomy: no transactions, no integrity constraints, no global

schema

Date

myDate yourDate

m(yourDate) = Datem(myDate) = Date


Emergent Semantics in PDMS

• P2P data management systems form (complex) mapping networks between models:Semantic Overlay Networks (SON)– Mappings manually or automatically generated– Mappings establish semantic correspondences– Mutually negotiated and verified (pragmatic dimension)

• Practical systems with the potential to exhibit emergent semantics properties

• Technical challenges?


Overview



Answering Queries in PDMS

• Semantic Query routing– To whom shall I forward a query posed against my local schema?

• Some (most) mappings will be (partially) faulty– Different views on conceptualizations– Low expressive power of mapping languages – Automatic schema matching techniques

• Alternatives– Local query resolution only: Low recall– Flooding the whole network (PDMS so far): Low precision

?


Analyzing PDMS Mapping Networks

• Standard deductive inference is not sufficient– Uncertainty on mappings and conceptualizations– Precision/Recall tradeoff

• Analyze Mapping Network– Feedback from query forwarding– Check for consistency– Abductive reasoning: find best possible explanation in case of

inconsistency

• Types of feedback– Query results: preservation of data dependencies, content

similarity– Transformed queries: Transitive closures of mapping operations

resulting in cycles and parallel paths


Example: Query Transformation Along Cycle

m0

m3

m4

q VS m3(m4(m0(q)))

art/Creator? VS art/creatDate?

q:art/Creator?

f0


Probabilistic Message Passing (Semantic Gossiping)

• Deriving quality measures for the mappings using feedback– Reduces uncertainty– Used to route query / optimize mappings

good bad

probably goodmaybe bad


Using the feedback

• The result of applying a composite mapping to a query should be identical to the original query for a cycle– Allows to estimate the probability of correctness of mapping

m0 m1

m2m3

m4m5

f0


Computing a Marginal for One Cycle

• P(m0, m3, m4, f0) = P(m0) P(m3) P(m4) P(f0 | m0, m3, m4)

• Determine P(mi | f0) given P(f0 | m0, m3, m4)?

• P(m0| f0)= ∑m3, m4 P(m0, m3, m4, f0) P(f0)-1

• But: feedbacks on different cycles are correlated– One wrong mapping will affect several cycles/paths– Need to express a global probabilistic model for the mapping graph

observedunknown


A Brief Intro to Factor-Graphs

• g(x1, x2, x3, x4) = fA(x1, x2)fB(x2, x3, x4)


Deriving PDMS Factor-Graphs

Innate probabilitiesof mappings being correct


PDMS Factor-Graphs

• Cyclic graph– Junction Tree?

• Centralization• Computational + communicational overhead

– Iterative Sum-Product • Corrrect only for tree structured networks• Approximate result

• How to perform iterative sum-product by message passing on the mapping graph?– Message passing in factor graph does not correspond to

connectivity of mapping graph– We want to rely on decentralized computations only


Embedded Message Passing

• Decentralized computations– Computationally inexpensive– Sums and Products

• Message-Passing Schedules– Lazy (piggybacking on query forwarding)

• No message overhead

– Periodic


Evaluation: Convergence

(undirected example graph, prior 0.7 delta 0.1)


Evaluation: Fault-tolerance (faulty links)

(undirected example graph, prior 0.8 delta 0.1)


Evaluation: Performance Detecting Errors

(randomly generated networks of 50 schemas and 200 mappings, TTL = 5)


Conclusion Probabilistic Message Passing

• A technique to implement emergent semantic processes– Decentralized decision making– Converges to an agreement on conceptualizations– Scalable and robust method to infer correct mappings in a semantic

overlay network

• Questions– Do such mapping networks exist?– What is their structure?


Overview



Semantic Overlay Networks

• Networks of schema mappings– Directed, weighted, redundant

• Semantic InteroperabilityTwo peers are said to be semantically interoperableif they can forward queries to each other in the mapping graph, potentially through series of translation links

• Question– Which are necessary conditions that a semantic overlay network

becomes semantically interoperable in the large-scale?• Idea: use percolation theory to detect the emergence of a

strongly connected component in S– Adaptation of a recent graph-theoretic framework (Newman,

Strogatz, Watts 2001)

SB SE

SC

SDSA SF

SG

0.1 0.7

0.8

0.9

0.2

0.1

0.6

1

0.7

1


The Model

• Large-scale semantic overlay networks as random graphs with arbitrary degree distribution

• Specificities of the model– Strong clustering (clustering coefficient cc)– Bidirectionality (bidirectionality coefficient bc)

• Based on generatingfunctionology(pk probability of degree k)

• Necessary condition for semantic interoperability in the large– Appearance of a giant strongly-connected component: ci > 0


Size of Out-component

links-in links-outstrongly

connectedcomponent

out-component

in-component

5000 10000 15000 20000# edges

0.2

0.4

0.6

0.8

Relative Size of Out−Component

b experimental

a theoretic


Evaluation: The Sequence Retrieval System (SRS)

• Bioinformatic libraries: EMBL, SwissProt, Prosite, etc.– Commercial information indexing and retrieval system– Links from one database to others by mapping identifiers– More than 380 databanks and

500 (undirected) links– Custom crawler


Results

• Connectivity indicator ci = 25.4– Giant connected component:

187 nodes

• Size of the giant component– 0.47 (derived)– 0.48 (observed)

• Powerlaw Topology• Small-World Graph

– Clustering coefficient = 0.32– Diameter = 9


Graphs with Same Power-law Degree Distribution

• Varying number of edges


Analyzing Weighted Networks

• Do we have a sufficient number of good mappings?• Using quality measures from the mappings derived from

message passing– Uniformly distributed weights between 0 and 1– Attribute / schema level

• Semantic query forwarding– Per-hop forwarding behaviors– Only forward if wi ≥ τ

• τ = 0 : flooding• τ = 1 : exact answers


Overview



GridVine: Annotating Shared Resources

• End-users create annotations / ”categories” / ”translation links”– Constraining the annotation mechanism: we do not expect them to

write ontologies, views…


GridVine: Annotating Shared Resources

• Principle of data independence– Scalable physical layer: structured overlay network (P2P network)– Semantic logical layer: Semantic Gossiping

Insert(key, value) Retrieve(key) Return(value)

Logical Layer(GridVine)

Physical Layer(P-Grid)

Insert(RDF triple)

Insert(RDF schema)

Insert(Schema translation)

SearchFor(query)

Return(tuples)


Mapping annotations onto P-Grid

00? 01? 10? 11?

0?? 1??

???

000 010 100 011

<rdf:Descriptionrdf:about="urn:x-pgrid:F59F92C8BC…lucene_green_100.gif">

<Year xmlns="pgrids://CF0C052CE4…Pdf.rdfs#">2001</Year></rdf:Description>

User-defined annotations (RDF triples)

<rdfs:Class rdf:ID="Pdf" rdfs:comment="New schema class"><rdfs:subClassOf rdf:resource="http://www.p-grid.org/p-

grid.rdfs#PGridDataFile"/></rdfs:Class><rdf:Property rdf:ID="Year"><rdfs:domain rdf:resource="#Pdf"/><rdfs:range

rdf:resource="http://www.w3.org/2001/XMLSchema#string"/></rdf:Property>

User-defined categories (RDFS)<Title xmlns="pgrids://CF0C052CE418FC78…PDFFIle.rdfs#">

<owl:equivalentProperty rdf:resource="pgrids://CF…Pdf.rdfs#Title" /><rdf:Statement>

<rdf:subject rdf:resource="pgrids://CF0…PDFFIle.rdfs#Title" /><rdf:predicate rdf:resource="pgrids://owl/hasCorrectness" /><rdf:object rdf:resource="1.0" />

</rdf:Statement></Title>

User-defined category translations (OWL)

⇒ RDQL queries

P-Grid

P-Grid: Structured overlay network•Supports key-based search•decentralized, scalable, self-organizing access structure


Traversals of the Semantic Overlay Network

• GridVine: structured P2P network = Distributed index– Query forwarding independent of structure of semantic overlay!

• Different query forwarding paradigms– Iterative forwarding– Recursive forwarding

2000 4000 6000 8000 10000 12000

20

40

60

80

100

% resultsreceived

[ms]time

recursive 60 p

iterative 60 p

recursive 15 p

iterative 15 p


Overview



Outlook

(Source: activecampus/UCSD)

Information Sharing in the Sensor Internet


Current Situation with Information Sharing

Different WSNs Discovery andcorrelation (difficult)Syntactic and semanticheterogeneity

Web publishing (repetitive)DB app, java app, Web interface, …

?


Challenges

• Provide generic and simple-to-use tools for publishing data collected from sensors over the Web– Data stream management

• Provide tools for discovering published sensor data– Semi-structured metadata

• Provide tools for correlating data from autonomous and heterogeneous sensor data sources– Emergent semantics


Publishing: Global Sensor Network

• A simple-to-use system to publish and correlate sensor data streams – Virtual sensors are sensors, sensor networks or derived data streams– GSN nodes managing virtual sensors

• Architecture– Virtual Sensors published in a P2P network using metadata annotations– GSN nodes connected in a peer-to-peer streaming network in the Web– Data processing specified in a temporal SQL extension


Development Status

Available through sourceforge: http://sourceforge.net/projects/globalsn


Discovery: PicShark

• Assume sensor data is being massively published– Already the case for images (photo sharing, Flickr!)

• Discovery depends on the availability of (structured) annotations– Content-based search capabilities are limited (no text!)

• Manual annotations are hard to obtain– Metadata scarcity

• Social annotation (folksonomies)


Exploiting Social Context

• Assume standardized annotation scheme– Attributes A1, …, An

• Information-theoretic measure of metadata scarcity of image I

• Reducing metadata scarcity by metadata propagation to similar images– Annotations from different members of a community– Similarity derived from metadata, features,

user relevance feedback

⎪⎩

⎪⎨⎧

=

−= ∑

)attributes# :(n otherwise n1

present is attribute if 0)(

))(log()()(

i

ii

Apwhere

ApApIH


PicShark Prototype

Extract existing metadata in different

formats

Extract features from image content and text annotations

Propagate metadata

Store metadata in standard formats in a peer-to-peer network


Summary Information Sharing

• Publishing and sharing of sensor data in a peer-to-peer architecture: Global Sensor Network

• Shared annotations of image/sensor data in a social network: PicShark

• Distributed reasoning on the correctness of mappings among heterogeneous annotation schemes: emergent semantics


Smart Earth


Acknowledgements

• Joint research with the following members and visitors of my group– Philippe Cudre Mauroux, Adriana Budura, Ali Salehi, Manfred

Hauswirth, Andras Feher, Tim van Pelt, Julien Gaugaz

• Funding provided by– Swiss National Foundation (SNF) through NCCR MICS– European Union through the Evergrow project (FET)


For more information

• www.mics.ch

• lsirwww.epfl.ch

• www.p-grid.org

• sourceforge.net/projects/globalsn

Thanks for your attention!

http://www.mics.ch/

http://www.p-grid.org/

Documents

Structure and Dynamics of Emergent Semantics Systemsnsfcac.rutgers.edu/conferences/icac/keynotes/icac_EmSem.pdfEmergent Semantics Systems Karl Aberer EPFL School of Computer and Communication