Knowledge Graph Embedding and 8 Knowledge Graph Embedding and Applications 15/4/2019 Why Knowledge Graph?

  • View
    0

  • Download
    0

Embed Size (px)

Text of Knowledge Graph Embedding and 8 Knowledge Graph Embedding and Applications 15/4/2019 Why Knowledge...

  • ADVANCED ANALYTICS INSTITUTE

    UNIVERSITY OF TECHNOLOGY SYDNEY

    feit.uts.edu.au

    dsmi.tech

    Knowledge Graph Embedding and Applications

    Zili Zhou, Qian Li, Guandong Xu, Shaowu Liu School of Software & Advanced Analytics Institute

    University of Technology Sydney Zili.Zhou@student.uts.edu.au

  • 2 Knowledge Graph Embedding and Applications 15/4/2019

    Agenda

    Concepts of Knowledge Graph (KG)

    Knowledge Graph Completion

    Knowledge Graph Embedding Methods

    Application of Knowledge Graph

    Conclusion and open research questions

  • 3 Knowledge Graph Embedding and Applications 15/4/2019

    Knowledge Graph

    • Knowledge graph (KG) is a large scale semantic network consisting of entities/concepts as well as the semantic relationships among them, which could be considered as a concise version of Semantic Web.

    • In the network (graph)

    • Each node = an entity; Each edge = a relation

    • A fact = RDF Triple (subject entity, relation, object entity)

    • For example, (Sydney, CityOf, Australia)

  • 4 Knowledge Graph Embedding and Applications 15/4/2019

    Web of Documents ➢ People can parse web of documents and

    extract information from them

    Knowledge Graph history

    Semantic Web (2007) ➢help machines to understand the web, so

    machines can help us to understand things.

    ➢ If machines have access to the data about

    things (i.e. knowledge) then they can do

    better job while processing documents

    Search Engines ➢ Google, Bing, Baidu …

  • 5 Knowledge Graph Embedding and Applications 15/4/2019

    RDF model

    ➢ In RDF knowledge

    always comes in three.

    ➢ RDF is a triple model

    i.e. every piece of

    knowledge is broken

    down into

    ➢ ( subject , predicate ,

    object )

  • 6 Knowledge Graph Embedding and Applications 15/4/2019

    • Yago, WordNet, Freebase, Probase, NELL, CYC, DBpedia, ….

    KG Evolving

    Date No. of KG

    2017-03-16 1,139

    2014-08-30 570

    2011-09-19 295

    2010-09-22 203

    2009-07-14 95

    2008-09-18 45

    2007-11-07 28

    2007-05-01 12

    "Linking Open Data cloud diagram 2017, by Andrejs Abele, John P. McCrae, Paul

    Buitelaar, Anja Jentzsch and Richard Cyganiak. http://lod-cloud.net/"

    http://lod-cloud.net/

  • 7 Knowledge Graph Embedding and Applications 15/4/2019

    Knowledge Graph essential

    • Some publicly available knowledge bases

    16

    Before 2000

  • 8 Knowledge Graph Embedding and Applications 15/4/2019

    Why Knowledge Graph?

    Why Knowledge Graph?

    1. Knowledge Graph has some advantages acting as an enhancement side information for application scenarios.

    2. Knowledge Graph has some special graph structure which causes challenges in Knowledge Graph Analysis area.

    - Multi-relational Network - Semantic Inference

    - Large scale - Semantically dense - High quality

  • 9 Knowledge Graph Embedding and Applications 15/4/2019

    KG advantage: Large scale

    KG # of entities / concepts # of facts

    DBpedia 4.58 million 3 billion

    Yago 10 million 120 million

    Probase 2.7 million 70 billion

    Freebase 44 million 2.4 billion

    0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 1,600,000

    persons

    places

    music albums

    films

    video games

    organizations

    species

    # of entities of specific domains in DBpedia

    # of entity in specific domain of dbpedia

    KG advantage: Large scale

  • 10 Knowledge Graph Embedding and Applications 15/4/2019

    KG advantage: Semantically dense

    • Quantitively Sparse item A, a repository of Github which is selected by precious few users, it is hard to do recommendation based limited number of historical user-item selection records.

    • However, the item is linked to an entity “TensorFlow”, and “TensorFlow” has dense semantic information in KG.

    • Through KG connection, another item B can be linked to item A, we may also recommend item A to a user who selected item B in historical user-item selection records.

    A

    B

    KG advantage: Semantically dense

  • 11 Knowledge Graph Embedding and Applications 15/4/2019

    KG advantage: High quality

    • Structured data

    • Unified format

    Unstructured

    Semi-structured

    Structured RDF Triples

    Subject Predicate Object

    TensorFlow Developer Google Brain

    TensorFlow WrittenIn Python

    Google Brain Location Mountain View, California

    Python Developer Python Software Foundation

    …...

    KG advantage: High quality

  • 12 Knowledge Graph Embedding and Applications 15/4/2019

    KG challenges: multi-relational network

    KG challenges: multi-relational network

    - Multiple types of edges - Large number of types

  • 13 Knowledge Graph Embedding and Applications 15/4/2019

    KG challenges: Semantic Inference

    KG challenges: semantic inference

    (a, r1, b) => (c, r1, d)

    KG related models needs to be reasonable for knowledge semantic inference

  • 14 Knowledge Graph Embedding and Applications 15/4/2019

    Knowledge Graph Completion

    Knowledge graph completion (KGC) • Given a particular 𝑟, for any entity pair (ℎ, 𝑡) such that (ℎ, 𝑟, 𝑡) ∉ 𝑂 judge whether ℎ and 𝑡 should

    be linked by 𝑟 or not.

    • Inferring missing facts from existing facts and external information.

    • Knowledge graph embedding is proposed to encode KG into low dimensional vector space

  • 15 Knowledge Graph Embedding and Applications 15/4/2019

    Representation Learning / Embedding (dimension reduction)

    • Discrete to Real-valued • Enable the symbolic data computing and generalization

    • Sparse to Dense • Reduce the influential of data sparsity

    • High dimension to Low dimension • Extract abstract latent information

    DeepWalk (Perozzi, AlRfou, and Skiena 2014)

    embedding

    Graph => Vector

    Subject to 1. Preserving graph structure 2. Low dimension

  • 16 Knowledge Graph Embedding and Applications 15/4/2019

    Knowledge Graph Embedding

    Knowledge graph embedding • Represent the entities and relations in a low-dimensional continuous vector space

    • Procedures

    • Represent entities/relations in a continuous vector space/operations (e.g., inner product)

    • Define a scoring function 𝑓𝑟(ℎ, 𝑡) on each fact (ℎ, 𝑟, 𝑡) to measure its plausibility

    • Estimate model parameters by maximizing the total plausibility of observed facts

    Entity Vector Encode Semantic Matching Model

    RESCAL [Nickel et al., 2011]

    DistMult [Yang et al., 2015]

    HolE [Nickel et al., 2016]

    Translational Distance Model

    TransE [Bordes et al., 2013]

    TransH [Wang et al., 2014]

    TransR [Lin et al., 2015]

    Two Models Types

    SME [Bordes et al., 2014]

    NTN [Socher et al., 2013]

    MLP [Dong et al., 2014]

    NAM [Liu et al., 2016]

  • 17 Knowledge Graph Embedding and Applications 15/4/2019

    Translational Distance Model: TransE , TransH, TransR

    • Exploit distance-based scoring functions that measures the plausibility of a fact as the distance between the two entities.

    Knowledge Graph Embedding

    Wang, Quan, et al. TKDE 2017

  • 18 Knowledge Graph Embedding and Applications 15/4/2019

    TransE

    • Entities and relations are embedded into same

    vector space.

    • Consider relation r as translation from entity h to

    entity t

    • Learning Assumption h+r=t

    Knowledge Graph Embedding

    Bordes, Antoine, et al. "Translating embeddings for modeling multi-relational data." Advances in neural information processing systems. 2013.

  • 19 Knowledge Graph Embedding and Applications 15/4/2019

    TransH: From original space to Hyperplane

    • TransH enables different roles of an entity in

    different relations.

    • Entities h and t are projected into specific

    hyperplane of relation r.

    • Then predict new links based on translation on

    hyperplane.

    Knowledge Graph Embedding

    Wang, Zhen, et al. "Knowledge graph embedding by translating on hyperplanes." Twenty-Eighth AAAI conference on artificial intelligence. 2014.

  • 20 Knowledge Graph Embedding and Applications 15/4/2019

    TransR:

    • TransR has similar idea with TransH.

    • Entities h and t are projected into

    specific subspace of relation r.

    • Predict new links based on translation in

    subspace.

    Knowledge Graph Embedding

    Lin, Yankai, et al. "Learning entity and relation embeddings for knowledge graph completion." Twenty-ninth AAAI conference on artificial intelligence. 2015.

  • 21 Knowledge Graph Embedding and Applications 15/4/2019

    Semantic Matching Model:

    • Exploit similarity-based scoring functions that measures plausibility of facts by matching

    latent semantics