The Future of Knowledge Graphs Moving beyond Simple ...qngw2014.bj.bcebos.com/upload/kg3/KG 2015 -...

The Future of Knowledge Graphs:Moving beyond

Simple Collections of Facts

Gerard de MeloAssistant Professor, Tsinghua University

http://gerard.demelo.org

The Future of Knowledge Graphs:Moving beyond

Simple Collections of Facts

Gerard de MeloAssistant Professor, Tsinghua University

http://gerard.demelo.org

25 Years of the World Wide Web:1989−2014

http://geekcom.wordpress.com/2009/03/19/

Tim Berners-Lee

HyperText(the “HT” in

“HTML”)

HyperText(the “HT” in

“HTML”)

Basic Idea:Connecting Data

Tim Berners-Lee

The Semantic WebThe Semantic Web

Tim Berners-Lee

col-league

born in Frankfurt

describedby

created by

A Web ofmachine-readableentity-relationshipdata

createdby

Frankfurt image: https://en.wikipedia.org/wiki/File:Frankfurt_Am_Main-Stadtansicht_von_der_Deutschherrnbruecke_am_fruehen_Abend-20110808.jpg

The Semantic WebThe Semantic Web

Tim Berners-Lee

A Web ofmachine-readableentity-relationshipdata

Frankfurt image: https://en.wikipedia.org/wiki/File:Frankfurt_Am_Main-Stadtansicht_von_der_Deutschherrnbruecke_am_fruehen_Abend-20110808.jpg

Problem:Just an academicidea, not used inpractice!

Big Knowledge CollectionsBig Knowledge Collections

Big Knowledge CollectionsBig Knowledge CollectionsBig Knowledge CollectionsBig Knowledge Collections

https://commons.wikimedia.org/wiki/File:Encyclopedia_Britannica_in_the_library_of_The_Kings_School,_Goa.jpg

Big Knowledge CollectionsBig Knowledge CollectionsBig Knowledge CollectionsBig Knowledge Collections

Rob Matthews: printed small sample of Wikipedia

Actually, a printedWikipedia corresponds to2000 Britannica volumes

Source:http://www.labnol.org/internet/wikipedia-printed-book/9136/

Actually, a printedWikipedia corresponds to2000 Britannica volumes

Source:http://www.labnol.org/internet/wikipedia-printed-book/9136/

Background: 2006/7Background: 2006/7

Image: weinbrenner.single.arabzadeh. Architektenwerkgemeinschafthttp://www.wsa-nt.de

Computer Science branchof Max Planck Society, whichhas had 33 Nobel Prize winners

Big Knowledge CollectionsBig Knowledge Collections

YAGO2. Hoffart et al. WWW 2011.

Gerard de Melo

YAGO 1: 2007 YAGO 1: 2007

Big Knowledge GraphsBig Knowledge Graphs

Gerard de Melo

Google Knowledge GraphGoogle Knowledge GraphGoogle Knowledge GraphGoogle Knowledge Graph

Gerard de Melo

Mainstream AdoptionMainstream AdoptionMainstream AdoptionMainstream Adoption

Solved:Solved:The Low-Hanging FruitThe Low-Hanging Fruit

What we should be able to askWhat we should be able to askWhat we should be able to askWhat we should be able to ask

Europe: Could be UK, France, Germany, etc.!Europe: Could be UK, France, Germany, etc.!

Spring: March, April, etc.Spring: March, April, etc.

Are any talks at KG 2015about social media?

The following talks at KG 2015seem related to social media:

...Image: Microsoft Cortana

Which talks at KG 2015 are mostrelated to my current projects?

The following talks at KG 2015seem most related to your work:

...Image: Microsoft Cortana

Beyond Collections of FactsBeyond Collections of Facts

1. Knowledge Integration and Organization

2. Natural Language Coupling

3. Multimodal and Cognitive Knowledge

The Web of DataThe Web of Data

LinkedData

The Web of DataThe Web of Data

LinkedIntegrated

Linked Data Is Still NotIntegrated Data

Technical Challenges

● Different access mechanisms:Individual HTTP requests, SPARQL endpoints,Downloads

● Downloads often not available

● Servers offline

Technical Challenges

● Different access mechanisms:Individual HTTP requests, SPARQL endpoints,Downloads

● Downloads often not available

● Servers offline

https://commons.wikimedia.org/wiki/File:Gov%27t_Shutdown,_Locked.._-_Flickr_-_daveynin.jpg

A lot of dataA lot of datais locked awayis locked awayto some extentto some extent

Semantic Challenges

● Different ontologies/schemas/vocabularies

● Links are incomplete

● Links are bad

● Data is sometimes bad or outdated

Semantic Challenges

● Different ontologies/schemas/vocabularies

● Links are incomplete

● Links are bad

● Data is sometimes bad or outdated

Disconnected InformationDisconnected Information

IsA ΦPhilosopher

RudolfCarnap

Studied at Universityof

BerlinCarnap

Was Professor at

Einstein

bornInAlbert Einstein HU Berlin

Current SituationCurrent SituationCurrent SituationCurrent Situation

Alexandre Duret-Lutz https://www.flickr.com/photos/gadl/110845690/

Goal: Integrated andOrganized KnowledgeGoal: Integrated and

Organized Knowledge

Theological Hall, Strahov Monastery Library, Prague

EntityIntegration

Entity IntegrationEntity Integration

Source: Bordes & Gabrilovich. KDD 2014 Tutorial

Source: Peter Mika

Entity Integration:Challenges

Entity-Level Integration

Gerard de Melo

Language-specific,Language-specific,Domain-specific,Domain-specific,Arbitrary DatabasesArbitrary Databases

Entity Integration:Computing Similarity Scores

Are these two the same? Are these two the same?

Compare attributes, relations, textCompare attributes, relations, text→ Similarity score (weighted link)→ Similarity score (weighted link)

Are these two the same? Are these two the same?

Compare attributes, relations, textCompare attributes, relations, text→ Similarity score (weighted link)→ Similarity score (weighted link)

ACL 2010AAAI 2013ACL 2010AAAI 2013

UseIdentity Linksto connectWhat isequivalent

Merging Structured DataMerging Structured DataMerging Structured DataMerging Structured Data

Merging Structured DataMerging Structured Data

Trentino Trentino-Alto Adige

One bad link is One bad link is enough to make aenough to make aconnected component connected component inconsistentinconsistent

OptimizationOptimizationProblem:Problem:

Make clustersMake clustersconsistentconsistent

OptimizationOptimizationProblem:Problem:

Make clustersMake clustersconsistentconsistent

Either by deletingEither by deletingedges or mergingedges or merging

conceptsconcepts

Either by deletingEither by deletingedges or mergingedges or merging

conceptsconcepts

Min. cost solution:Min. cost solution:NP-hardNP-hard

APX-hardAPX-hard

Min. cost solution:Min. cost solution:NP-hardNP-hard

APX-hardAPX-hard

Finally, use region growingFinally, use region growingalgorithm in the spiritalgorithm in the spirit

of Leighton & Rao 1988of Leighton & Rao 1988

Finally, use region growingFinally, use region growingalgorithm in the spiritalgorithm in the spirit

of Leighton & Rao 1988of Leighton & Rao 1988

Linear Program RelaxationLinear Program RelaxationLinear Program RelaxationLinear Program Relaxation

Approximation Guarantee:Approximation Guarantee:4ln(nq+1) 4ln(nq+1)

for n distinctness assertions,for n distinctness assertions,q=max |Dq=max |D

i,ji,j||

but independent of |Dbut independent of |Dii| !| !

Approximation Guarantee:Approximation Guarantee:4ln(nq+1) 4ln(nq+1)

for n distinctness assertions,for n distinctness assertions,q=max |Dq=max |D

i,ji,j||

but independent of |Dbut independent of |Dii| !| !

Linear Program RelaxationLinear Program RelaxationLinear Program RelaxationLinear Program Relaxation

Nice:Nice:

This generalizes theThis generalizes theHungarian AlgorithmHungarian Algorithm

from bipartite matchingsfrom bipartite matchingsto many non-standard kindsto many non-standard kinds

of matchingsof matchings

(cf. de Melo. AAAI 2013)(cf. de Melo. AAAI 2013)

Nice:Nice:

This generalizes theThis generalizes theHungarian AlgorithmHungarian Algorithm

from bipartite matchingsfrom bipartite matchingsto many non-standard kindsto many non-standard kinds

of matchingsof matchings

(cf. de Melo. AAAI 2013)(cf. de Melo. AAAI 2013)

TaxonomicIntegrationTaxonomicIntegration

Question Answering

IBM's Jeopardy!-winning Watson system

Gerard de Melo

Question Answering

Gerard de Melo

Question Answering

Gerard de Melo

De Melo & Weikum (2010).CIKM Best Interdisciplinary Paper AwardDe Melo & Weikum (2010).CIKM Best Interdisciplinary Paper Award

Taxonomic Integration:Taxonomic Integration:MENTA ApproachMENTA Approach

Taxonomic Integration:Taxonomic Integration:MENTAMENTA

Image: https://de.wikipedia.org/wiki/Datei:Bersntol_palae.jpg

Fersental(Bersntol, Valle dei Mòcheni)Fersental(Bersntol, Valle dei Mòcheni)

https://de.wikipedia.org/wiki/Datei:Language_distribution_Trentino_2011.png

Fersental(Bersntol, Valle dei Mòcheni)Fersental(Bersntol, Valle dei Mòcheni)

Use Identity ConstraintAlgorithm to form equivalence classes

Markov Chain RandomWalk with Restartsto Rank Parents

KnowledgeIntegrationKnowledgeIntegration

Knowledge IntegrationKnowledge Integration

X isAuthorOf YY writtenBy XX wrote YY writtenInYear Z

UWN/MENTA

multilingual extension of WordNet forword senses and taxonomical information over 200 languages

Gerard de Melo

Search Interfaces

“Which companies were created during the last century in Silicon Valley ?”

YAGO2:WWW 2011

Best Demo Award

YAGO2:WWW 2011

Best Demo Award

Gerard de Melo

Example:Example:Lexvo.orgLexvo.orgExample:Example:Lexvo.orgLexvo.org

Semantic WebSemantic WebJournal 2014Journal 2014Semantic WebSemantic WebJournal 2014Journal 2014

InterdisciplinaryInterdisciplinaryWork, e.g. inWork, e.g. inDigital HumanitiesDigital Humanities

Lexvo.orgLexvo.org

YAGO-SUMO: KnowledgeIntegration for ReasoningYAGO-SUMO: KnowledgeIntegration for Reasoning

70000 axiomsopen source

YAGO-SUMO

SUMO's axiomsattached to

real-world instances,

overall many millions of axioms

YAGO-SUMO

SUMO's axiomsattached to

real-world instances,

overall many millions of axioms

Smarter Question AnsweringSmarter Question Answering

Sutcliffe et al.Sutcliffe et al.Sutcliffe et al.Sutcliffe et al.

Gerard de Melo

What is the capital of New Jersey?

Question AnsweringQuestion Answering

Buy Jaguar in Beijing

Which meaning?Which meaning?

JaguarJaguar

Buy Jaguar in Beijing

“Do Macs now supportWindows?”

Lexical Knowledge Bases

Multilingual Lexical Knowledge

UWN (de Melo & Weikum 2009)

UWN: Universal Wordnet

Gerard de Melo

Exploit millionsof translationsfrom the Web

UWN: “Universal Wordnet”

Learn regularized regression model

to predict edge weightsfor a given pair of nodes

Learn regularized regression model

to predict edge weightsfor a given pair of nodes

Graph-based features, weighted by various edge weight metrics

Gerard de Melo

CIKM 2009CIKM 2009

UWN: Universal Wordnet

over 1,000,000 words in over 100 languages

CIKM 2009CIKM 2009 ICGL 2008ICGL 2008Best Paper AwardBest Paper AwardICGL 2008ICGL 2008Best Paper AwardBest Paper Award

Gerard de Melo

UWN: Getting StartedUWN: Getting Started

Simple API for JVM Languages

val uwn = new UWN(new File("plugins/"))for (m <- uwn.getMeanings("souris", "fra"))

println(m)

Or Just Download the TSV File

Simple API for JVM Languages

val uwn = new UWN(new File("plugins/"))for (m <- uwn.getMeanings("souris", "fra"))

println(m)

Or Just Download the TSV File

http://lexvo.org/uwn/

UWNUWNUWNUWN

http://www.lexvo.org/uwn/ http://www.lexvo.org/uwn/ http://www.lexvo.org/uwn/ http://www.lexvo.org/uwn/

UWNUWNUWNUWN

LREC 2014LREC 2014

Etymological WordnetEtymological Wordnet

Real Understanding?Real Understanding?

Knowledge Bases keep growing, butmuch of the Web is still not truly understood

Equivalent:

MetaWeb was acquired by Google.MetaWeb was just recently acquired by Google.MetaWeb, surprisingly, was acquired by Google.

Relation UnderstandingRelation Understanding

MetaWeb was bought out by Google.Google bought MetaWeb.Google acquired MetaWeb.MetaWeb was sold to Google.Google's acquisition of MetaWeb.Google's MetaWeb acquisition.and so on...

Underlying frame: Commercial transfer

Capture the “who-did-what-to-whom”

Microsoft bought the patent from Nokia.Nokia sold the patent to Microsoft.The patent was acquired by Microsoft [from Nokia].The patent was sold [by Nokia] to Microsoft.

Relation Understanding:N-Ary Relations

Buyer: Microsoft

Seller: Nokia

Product: The patent

Relation IntegrationRelation Integration

YAGO: isMarriedTo predicateYAGO: isMarriedTo predicate

Freebase: Marriage EntityFreebase: Marriage Entity

Challenge:Modelling

Differences

Challenge:Modelling

Differences

Relation Integration:FrameBase.org

Bringing knowledge into a standard formbased on natural language (FrameNet)

Background:Berkeley FrameNet Project

MENTA: MediaMENTA: Media

Knowlywood: Human ActivitiesKnowlywood: Human Activities

CIKM 2015CIKM 2015

Knowlywood: Human ActivitiesKnowlywood: Human Activities

CIKM 2015CIKM 2015

Learning Common-SenseLearning Common-SenseLearning Common-SenseLearning Common-Sense

WebChild

AAAI 2014WSDM 2014AAAI 2011

WebChild

AAAI 2014WSDM 2014AAAI 2011

Online: http://bit.ly/webchild

Learning Common-SenseLearning Common-SenseLearning Common-SenseLearning Common-Sense

WebChild

(Tandon et al.WSDM 2014):

WebChild

(Tandon et al.WSDM 2014):

Comparative KnowledgeComparative KnowledgeComparative KnowledgeComparative Knowledge

AAAI 2014AAAI 2014

Joint Disambiguation:<bullet train, quicker than, bus>

<train, slower than, plane><plane, faster than, train><bus, slower than, plane>

Joint Disambiguation:<bullet train, quicker than, bus>

<train, slower than, plane><plane, faster than, train><bus, slower than, plane>

ImpLex:Implicative Commitments

Netflix admitted that they had developed

a prototype.

VSoft pretended that they had developed

a prototype.

Did theydevelop a

prototype?

Gerard de Melo

de Melo & de Paiva (2014)based onLauri Kartunnen's work

ImpLex:Implicative Commitments

It is confusing thatApple has acquired

the technology.

It is unlikely thatApple has acquired

the technology.

Have theyacquired it?

de Melo &de Paiva 2014

Word VectorsWord VectorsWord VectorsWord Vectors

x xpetronia

sparrow

parched

aridxdry

x bird

http://www.wikihow.com/Read-a-Book-to-a-Baby-or-Infant#/Image:Read-a-Book-to-a-Baby-or-Infant-Step-5.jpg

Why are Why are Word Vectors Successful?Word Vectors Successful?

Matej Kren: Idiom. Prague Municipal Libraryhttps://www.flickr.com/photos/ill-padrino/6437837857/

● Fast Training on Large Amounts of Text● Joint Optimization (Mutual Influence

between Word Vectors)

● Fast Training on Large Amounts of Text● Joint Optimization (Mutual Influence

between Word Vectors)

Jiaqiang Chen and Gerard de Melo 2015

What Goes into Word Vectors?What Goes into Word Vectors?What Goes into Word Vectors?What Goes into Word Vectors?

The Roman Empire was remarkablymulticultural, with ”a rather astonishingcohesive capacity” to create a senseof shared identity while encompassingdiverse peoples within its political system over a long span of time.

syntactic

syntactic semantic!

semantic!syntactic syntactic?

semantic!syntactic syntactic? ?

semantic!syntactic ?

Word2Vec Solution:Subsampling

syntactic?

semantic!syntactic ?syntactic?

Textual Big DataTextual Big DataTextual Big DataTextual Big Data

Image: https://500px.com/photo/96738645/street-bookstore-by-pablo-tarrero

Better Word Vectorsby Exploiting Knowledge

BetterWord Embeddings

Joint Training

Best Paper Awardat NAACL 2015Vector Space

Modeling Workshop

Best Paper Awardat NAACL 2015Vector Space

Modeling Workshop

Textual Big Data Knowledge

Multilingual Word Vectors

Combine word2vec/GloVe Vectorswith multilingual lexical data from Wiktionary

Gerard de Melo

Available soon athttp://gerard.demelo.org/representations/

Contact me!

Multilingual Word Vectors

Combine word2vec/GloVe Vectorswith multilingual lexical data from Wiktionary

Gerard de Melo

Contact me!

Multilingual Analogical ReasoningMultilingual Analogical ReasoningMultilingual Analogical ReasoningMultilingual Analogical Reasoning

Coffee is to Starbucksas ... is to Lipton?

System: Tea

Source vectors: GLOVE

What we should be able to getWhat we should be able to getin the futurein the future

What is thisall about?

This is about the US budget debt ceiling,which needs to be extended. RepublicanSenate majority leader Mitch McConell

would like to...

Image: Microsoft Cortana

Huffington Post

Summary

Knowledge Integration and Organization► MENTA taxonomy► Lexvo.org Linked Data

Natural Language Coupling► Universal WordNet► FrameBase.org

Multimodal and Cognitive Knowledge► Knowlywood► WebChild

More Information:

www.demelo.orggdm@demelo.org

More Information:

www.demelo.orggdm@demelo.org

Gerard de Melo

The Future of Knowledge Graphs Moving beyond Simple ...qngw2014.bj.bcebos.com/upload/kg3/KG 2015 -...

Documents

· 49 Ke4 Rel; 50 Kd4 Rgl; 51 Rc6 Kf5; 52 Rc5 Kf6; 53 Rg5 g6; 54 Kc5 1-0. Craig Mar — Rich Carlstadt ... 51 Kg3 Kd3; 52 Kf2 Ke4; 53 Kg; Craig Mar - Ken Walters

Caraerizzazionedicelluleimmunosoppressorie ... 14 NOVEMBRE/Castella.ppt.pdf · “MyeloidRderived(suppressor(cells(as(regulators(of(the(immune(system”,(Gabrilovich(DI,(Nagaraj(S,((NatRev(Immunol,!2009!Mar;9(3):162@74

Employing External Rich Knowledge for Machine …qngw2014.bj.bcebos.com/upload/2016/04/王炳宁-Employing...Employing External Rich Knowledge for Machine Comprehension 中国科学院自动化研究所

基于路径的知识表示与推理qngw2014.bj.bcebos.com/upload/2016/04/林衍凯-基于...3 American_University Punahou_School 4 University_of_Michigan University_of_Chicago 5 Columbia_University

Jianyong Wang Department of Computer Science and ...qngw2014.bj.bcebos.com/upload/kg3/KG 2015 - Entity Linking wih a... · Jianyong Wang Department of Computer Science and Technology

Reading Sessions KG3-Barabschools.edu.sa/wp-content/uploads/2020/04/KG3... · Teacher Ms.Inas Ghalawan Ms.Inas Ghalawan Ms.Inas Ghalawan Link 0270153 0270153 0270153 Arabic Session

AL NOOR INTERNATIONAL SCHOOL Weekly Plan and Assignments …alnoor.edu.sa/userfiles/2017/11/pages/(27)--KG3 Regular... · 2017. 11. 9. · AL NOOR INTERNATIONAL SCHOOL Riyadh, Saudi

gabrilovich.comgabrilovich.com/publications/papers/Bilenko2009IRA.pdf · i Organizing Committee Mikhail Bilenko Microsoft Research Evgeniy Gabrilovich Yahoo! Research Matthew Richardson

Knowledge-Based Trust: Estimating the Trustworthiness of ... · Knowledge-Based Trust: Estimating the Trustworthiness of Web Sources Xin Luna Dong, Evgeniy Gabrilovich, Kevin Murphy,

Segment-Level Sequence Tagging using Gated Recursive Semi ...qngw2014.bj.bcebos.com/upload/2016/06/17-卓靖炜.pdf · Neighbour Predrag Mijatovic Red Cross South African Breweries

HOLISTIC DATA INTEGRATION FOR BIG DATA - uni-leipzig.deNickel, Murphy, Tresp, Gabrilovich: A review of rela tional machine learning for knowledge graphs (Arxiv, 2015) uniform representation

December 5, 2004 Seminar on Aspect-Oriented Software Development (236800) Static Analysis of Aspects Evgeniy Gabrilovich gabr

qngw2014.bj.bcebos.comqngw2014.bj.bcebos.com/upload/kg3/KG 2015... · Ì ÉÊË¢h)ÌÍ ÎÏ\vÐÑ ÒÓ Ô ÕÖãv7Zd ×[ØÉÙÚ\ûíÛ¼Ü ^Ý_Þ +Û ä Ð 3 & 7= ># &$3 lN ß\à

KG3 math 425, 426

78K0R/KG3 16-bit Single-Chip Microcontrollers UD · In the case of a device that uses different power supplies for the internal operation and ... NEC Electronics assumes no responsibility

Crowd Scene Understanding with Coherent Recurrent Neural ...qngw2014.bj.bcebos.com/upload/2016/05/dongyinpeng.pdf · Crowd scene is the scene of public places where a large group

KG3...numbers from50-100 p:8,9,10,,11,26,27 from text book number names10-30 after numbers 1-100 before1-20 ball in the basket passing the parcel sept phonic words- et,en,ed,eg,ut,um

Text Mining Tutorial – Part IIcci.drexel.edu/bigdata/bigdata2016/files/Tutorial1-2.pdf · 2016-12-12 · Text Mining Tutorial ... Gabrilovich & Markovitch IJCAI-2007. How to categorize

6 1 £ © * ï ¼qngw2014.bj.bcebos.com/upload/2016/04/刘康... · grammars,” in Proceedings of the 21st Uncertainty in Artiﬁcial Intelligence, 2005, pp. 658–666. [Clarke,

i, Poet: Automatic Poetry Composition through Recurrent Neural …qngw2014.bj.bcebos.com/upload/2016/05/yanrui-ipoet.pdf · 2016-05-23 · – Poems of Tang Dynasty – Poems of Song