@Sojoner

ENTITY LINKING IN FOR FUN (CLOJURE)

AGENDA★ Motivation

★ Entity linking

★ Reflector overview

★ Represent the data

★ A bit code

★ Learnings

MOTIVATION★ practicing Clojure

backend & frontend

★ using core.async

★ learning about FP

★ Gödel-Escher-Bach like information revealer

★ How will the Wikipedia reflect a given text when thrown against it ?

★ is the task of determining the identity of entities mentioned in text […] Wikipedia

★ You can improve search experience by enriching documents with entities e.g facet search for persons locations a.o

ENTITY LINKING& SEARCH

ENTITY LINKINGSYSTEMS & CHALLENGES

Simple stuff ;-)

1. spotting

2. searching

3. disambiguation

4. prune irrelevant stuff

REFLECTOR OVERVIEW& TECHNOLOGIES

REFLECTOR OVERVIEW& TECHNOLOGIES

REFLECTOR OVERVIEW& TECHNOLOGIES

REFLECTOR OVERVIEW& SYSTEM

titles

articles

categories

back frontstorage

REPRESENT THE DATA& DATA EXAMPLES

REPRESENT THE DATA& DATA EXAMPLES

AMBIGUOUS UNIQUE

A BIT CODE& BACK

A BIT CODE& BACK

★ main starts components

★ http is the external interface

★ communicator sends massages

★ linker does entity linking with solr

★ search searches pages in ES

★ wikigraph computes paths

★ switch glues the core.async channels together

A BIT CODE& BACK

A BIT CODE& BACK

A BIT CODE& BACK

A BIT CODE& BACK

A BIT CODE& BACK

A BIT CODE& FRONT

★ core handles communication and app state

★ ui OM components

★ channels just the definitions

A BIT CODE& FRONT

A BIT CODE& FRONT

A BIT CODE& FRONT

A BIT CODE& FRONT

LEARNINGS& FIN

★ Structure clojure code via components

★ Same language full stack

★ Async communication from front to back

★ use favorite datastores with clojure

★ learn about wikipedia

SHOULDERS& FIN

https://github.com/OpenSextant/SolrTextTagger

https://github.com/stuartsierra/component

https://github.com/clojure/core.async

https://github.com/swannodette/om

https://github.com/elasticsearch/elasticsearch-river-wikipedia

https://github.com/matthiasn/BirdWatch