Download pdf - Big Data & Cloud | An Introduction to Neo4j (and Doctor Who) | Ian Robinson

An Introduc+on to Doctor Who (and Neo4j)

@iansrobinson

#neo4j

#neo4j

Neo4j is a Graph Database

#neo4j

Property Graph

#neo4j

Property Graph

#neo4j

Neo4j

#neo4j

32 billion nodes 32 billion rela+onships 64 billion proper+es

#neo4j

Community

Advanced

Enterprise

stole from

loves loves

enemy

enemy A Good Man Goes to War

appeared in

appeared in

appeared in

appeared in

Victory of the Daleks

appeared in

appeared in

companion

companion

enemy

GraphDatabaseService db = new EmbeddedGraphDatabase("/data/drwho");


Node theDoctor = db.createNode(); theDoctor.setProperty("name", "The Doctor");

Node daleks = db.createNode(); daleks.setProperty("name", "Daleks"); Node cybermen = db.createNode(); cybermen.setProperty("name", "Cybermen");




theDoctor.createRelationshipTo(daleks, DynamicRelationshipType.withName("enemy")); theDoctor.createRelationshipTo(cybermen, DynamicRelationshipType.withName("enemy"));

enemy

enemy

GraphDatabaseService db = new EmbeddedGraphDatabase("/data/drwho"); Transaction tx = db.beginTx(); try {



theDoctor.createRelationshipTo(daleks, DynamicRelationshipType.withName("enemy")); theDoctor.createRelationshipTo(cybermen, DynamicRelationshipType.withName("enemy"));

tx.success();

} finally { tx.finish();

}

enemy

enemy

#neo4j

How do I query the data?

hOp://opfm.jpl.nasa.gov/

hOp://news.xinhuanet.com

#neo4j

Dalek Props hOp://www.dalek6388.co.uk/

!tle:Power of the Daleks

species:Dalek

APPEARED_IN

USED_IN props:Daleks

name:Dalek Six-‐5

name:Dalek 7 name:Dalek 2 name:Dalek 1

MEMBER_OF MEMBER_OF

type:shoulders type:skirt

name:Dalek Six-‐5

name:Dalek 6 name:Dalek 5

ORIGINAL_PROP ORIGINAL_PROP

ORIGINAL_PROP ORIGINAL_PROP

name:Dalek 1 name:Dalek 2 name:Dalek 7

!tle:Power of the Daleks !tle:The

Daleks

!tle:The Dalek

Invasion of Earth

name:Dalek Two-‐1 name:Dalek One-‐5 name:Dalek Six-‐7

name: Dalek Six-‐5 name:Dalek 1 name:Dalek 2 name:Dalek 7

#neo4j

Supply Chain Traceability

#neo4j

Traversal Framework

•  Visits (and returns) nodes based on traversal descrip+on

•  Powerful; can customize: – Rela+onships followed – Branch selec+on policy – Node/rela+onship uniqueness constraints – Path evaluators

Node theDaleks = database.index().forNodes("species").get("name", "Dalek").getSingle(); Traverser traverser = Traversal.description() .depthFirst() .relationships(Rels.APPEARED_IN, Direction.OUTGOING) .relationships(Rels.USED_IN, Direction.INCOMING) .relationships(Rels.MEMBER_OF, Direction.INCOMING) .relationships(Rels.COMPOSED_OF, Direction.OUTGOING) .relationships(Rels.ORIGINAL_PROP, Direction.OUTGOING) .evaluator(new Evaluator() { @Override public Evaluation evaluate(Path path) { if (path.lastRelationship() != null && path.lastRelationship().isType(Rels.ORIGINAL_PROP)){ return Evaluation.INCLUDE_AND_PRUNE; } return Evaluation.EXCLUDE_AND_CONTINUE; } }) .uniqueness(Uniqueness.NONE) .traverse(theDaleks); Description

Node theDaleks = database.index().forNodes("species").get("name", "Dalek").getSingle(); Traverser traverser = Traversal.description() .depthFirst() .relationships(Rels.APPEARED_IN, Direction.OUTGOING) .relationships(Rels.USED_IN, Direction.INCOMING) .relationships(Rels.MEMBER_OF, Direction.INCOMING) .relationships(Rels.COMPOSED_OF, Direction.OUTGOING) .relationships(Rels.ORIGINAL_PROP, Direction.OUTGOING) .evaluator(new Evaluator() { @Override public Evaluation evaluate(Path path) { if (path.lastRelationship() != null && path.lastRelationship().isType(Rels.ORIGINAL_PROP)){ return Evaluation.INCLUDE_AND_PRUNE; } return Evaluation.EXCLUDE_AND_CONTINUE; } }) .uniqueness(Uniqueness.NONE) .traverse(theDaleks); Index Lookup

Node theDaleks = database.index().forNodes("species").get("name", "Dalek").getSingle(); Traverser traverser = Traversal.description() .depthFirst() .relationships(Rels.APPEARED_IN, Direction.OUTGOING) .relationships(Rels.USED_IN, Direction.INCOMING) .relationships(Rels.MEMBER_OF, Direction.INCOMING) .relationships(Rels.COMPOSED_OF, Direction.OUTGOING) .relationships(Rels.ORIGINAL_PROP, Direction.OUTGOING) .evaluator(new Evaluator() { @Override public Evaluation evaluate(Path path) { if (path.lastRelationship() != null && path.lastRelationship().isType(Rels.ORIGINAL_PROP)){ return Evaluation.INCLUDE_AND_PRUNE; } return Evaluation.EXCLUDE_AND_CONTINUE; } }) .uniqueness(Uniqueness.NONE) .traverse(theDaleks); Branching

Node theDaleks = database.index().forNodes("species").get("name", "Dalek").getSingle(); Traverser traverser = Traversal.description() .depthFirst() .relationships(Rels.APPEARED_IN, Direction.OUTGOING) .relationships(Rels.USED_IN, Direction.INCOMING) .relationships(Rels.MEMBER_OF, Direction.INCOMING) .relationships(Rels.COMPOSED_OF, Direction.OUTGOING) .relationships(Rels.ORIGINAL_PROP, Direction.OUTGOING) .evaluator(new Evaluator() { @Override public Evaluation evaluate(Path path) { if (path.lastRelationship() != null && path.lastRelationship().isType(Rels.ORIGINAL_PROP)){ return Evaluation.INCLUDE_AND_PRUNE; } return Evaluation.EXCLUDE_AND_CONTINUE; } }) .uniqueness(Uniqueness.NONE) .traverse(theDaleks); Relationships

Node theDaleks = database.index().forNodes("species").get("name", "Dalek").getSingle(); Traverser traverser = Traversal.description() .depthFirst() .relationships(Rels.APPEARED_IN, Direction.OUTGOING) .relationships(Rels.USED_IN, Direction.INCOMING) .relationships(Rels.MEMBER_OF, Direction.INCOMING) .relationships(Rels.COMPOSED_OF, Direction.OUTGOING) .relationships(Rels.ORIGINAL_PROP, Direction.OUTGOING) .evaluator(new Evaluator() { @Override public Evaluation evaluate(Path path) { if (path.lastRelationship() != null && path.lastRelationship().isType(Rels.ORIGINAL_PROP)){ return Evaluation.INCLUDE_AND_PRUNE; } return Evaluation.EXCLUDE_AND_CONTINUE; } }) .uniqueness(Uniqueness.NONE) .traverse(theDaleks); Evaluator

Node theDaleks = database.index().forNodes("species").get("name", "Dalek").getSingle(); Traverser traverser = Traversal.description() .depthFirst() .relationships(Rels.APPEARED_IN, Direction.OUTGOING) .relationships(Rels.USED_IN, Direction.INCOMING) .relationships(Rels.MEMBER_OF, Direction.INCOMING) .relationships(Rels.COMPOSED_OF, Direction.OUTGOING) .relationships(Rels.ORIGINAL_PROP, Direction.OUTGOING) .evaluator(new Evaluator() { @Override public Evaluation evaluate(Path path) { if (path.lastRelationship() != null && path.lastRelationship().isType(Rels.ORIGINAL_PROP)){ return Evaluation.INCLUDE_AND_PRUNE; } return Evaluation.EXCLUDE_AND_CONTINUE; } }) .uniqueness(Uniqueness.NONE) .traverse(theDaleks); Uniqueness

Node theDaleks = database.index().forNodes("species").get("name", "Dalek").getSingle(); Traverser traverser = Traversal.description() .depthFirst() .relationships(Rels.APPEARED_IN, Direction.OUTGOING) .relationships(Rels.USED_IN, Direction.INCOMING) .relationships(Rels.MEMBER_OF, Direction.INCOMING) .relationships(Rels.COMPOSED_OF, Direction.OUTGOING) .relationships(Rels.ORIGINAL_PROP, Direction.OUTGOING) .evaluator(new Evaluator() { @Override public Evaluation evaluate(Path path) { if (path.lastRelationship() != null && path.lastRelationship().isType(Rels.ORIGINAL_PROP)){ return Evaluation.INCLUDE_AND_PRUNE; } return Evaluation.EXCLUDE_AND_CONTINUE; } }) .uniqueness(Uniqueness.NONE) .traverse(theDaleks); Traverse

Iterable<Path> path.startNode() path.endNode()

APPEARED_IN USED_IN COMPOSED_OF MEMBER_OF ORIGINAL_PROP

species:Dalek +tle:Power of the Daleks props:Daleks name:Dalek 7 type:skirt name:Dalek 7


species:Daleks +tle:Power of the Daleks props:Daleks name:Dalek 7 type:shoulder name:Dalek 7


species:Daleks +tle:Power of the Daleks props:Daleks name:Dalek Six-‐5 type:skirt name:Dalek 5

#neo4j

Cypher

•  Declara+ve graph pa8ern matching language – Humane regular expressions for graphs

•  New in 1.4 – Con+nuing, rapid feature development

•  Supports queries –  Including aggrega+on, ordering and limits – Muta+ng opera+ons coming in 1.6

start daleks=node:species(species='Dalek') match daleks-‐[:APPEARED_IN]-‐>episode<-‐[:USED_IN]-‐ ()<-‐[:MEMBER_OF]-‐()-‐[:COMPOSED_OF]-‐> part-‐[:ORIGINAL_PROP]-‐>originalprop return originalprop.name, part.type, count(episode.Qtle) order by count(episode.Qtle) desc limit 1

Cypher Query


Index Lookup


Match Nodes & Relationships

match daleks-‐[:APPEARED_IN]-‐>episode<-‐[:USED_IN]-‐ ()<-‐[:MEMBER_OF]-‐()-‐[:COMPOSED_OF]-‐> part-‐[:ORIGINAL_PROP]-‐>originalprop

daleks episode

part originalprop

APPEARED_IN

USED_IN

MEMBER_OF

COMPOSED_OF

ORIGINAL_PROP


Return Values

CypherParser parser = new CypherParser(); ExecuQonEngine engine = new ExecuQonEngine(universe.getDatabase()); String cql = "start daleks=node:species(species='Dalek')" + " match daleks-‐[:APPEARED_IN]-‐>episode<-‐[:USED_IN]-‐()<-‐[:MEMBER_OF]-‐()" + "-‐[:COMPOSED_OF]-‐>part-‐[:ORIGINAL_PROP]-‐>originalprop" + " return originalprop.name, part.type, count(episode.Qtle)" + " order by count(episode.Qtle) desc limit 1"; Query query = parser.parse(cql); ExecuQonResult result = engine.execute(query); String originalProp = result.javaColumnAs("originalprop.name").next().toString(); String part = result.javaColumnAs("part.type").next().toString(); int episodeCount = (Integer) result.javaColumnAs("count(episode.Qtle)").next();

In Code


Setup


Parse & Execute


Results

In Webadmin

#neo4j

The Hardest Working Prop Part

hOp://www.dalek6388.co.uk/

Dalek One’s shoulders

#neo4j

Download

hOp://neo4j.org/download/ hOp://www.springsource.org/spring-‐data/neo4j

#neo4j

Tutorial

hOps://github.com/jimwebber/neo4j-‐tutorial

#neo4j

Dematerialize…

@iansrobinson [email protected]

1.   Subtract: Select rectangle then 3 black triangles

2.   Union: All


2.   Union: All


2.   Union: All

daleks episode

part originalprop

match (daleks)-‐[:APPEARED_IN]-‐>(episode)<-‐[:USED_IN]-‐ ()<-‐[:MEMBER_OF]-‐()-‐[:COMPOSED_OF]-‐> (part)-‐[:ORIGINAL_PROP]-‐>(originalprop)

APPEARED_IN

USED_IN

MEMBER_OF

COMPOSED_OF

ORIGINAL_PROP