Graph Intuitive Query Language for Relational …jindal-web.appspot.com/slides/IEEEBigData.pdfIEEE...

Alekh Jindal

GraphiQLGraph Intuitive Query Language for Relational Databases

Sam Madden

Mike Stonebraker

Amol Deshpande

MITUniversity

of Maryland

IEEE BigData 2014Talking onat

Supervisors

collaborate

work sabbatical

Alekh Jindal

Sam Madden

Mike Stonebraker

Amol Deshpande

MITUniversity

of Maryland

Supervisors

collaborate

work sabbatical

Relational Database

Expensive!

Graph Analysis =

Graph Algorithms

StoreExtractPreprocessUpdateFailoverPostprocess

Graph Analysis =

Graph Algorithms

StoreExtractPreprocessUpdateFailoverPostprocess

+“Counting Triangles with Vertica”

“Scalable Social Graph Analytics Using the Vertica Analytic Platform,”

“Graph Analysis: Do We Have to Reinvent the Wheel?”

“Query Optimization of Distributed Pattern Matching,”

“GraphX: A Resilient Distributed Graph System on Spark,”

“Vertexica: Your Relational Friend for Graph Analytics!”

Relational DatabaseRelational Database

Problem !

Alekh Jindal

Sam Madden

Mike Stonebraker

Amol Deshpande

MITUniversity

of Maryland

Supervisors

collaborate

work sabbatical

Alekh Jindal

Sam Madden

Mike Stonebraker

Amol Deshpande

MITUniversity

of Maryland

Supervisors

collaborate

work sabbatical

SELECT

UPDATEFROM

GROUP BY

Redundant EffortAlekh Jindal

Sam Madden

Mike Stonebraker

Amol Deshpande

MITUniversity

of Maryland

Supervisors

collaborate

work sabbatical

Alekh Jindal

Sam Madden

Mike Stonebraker

Amol Deshpande

MITUniversity

of Maryland

Supervisors

collaborate

work sabbatical

Alekh Jindal

Sam Madden

Mike Stonebraker

Amol Deshpande

MITUniversity

of Maryland

Supervisors

collaborate

work sabbatical

Alekh Jindal

Sam Madden

Mike Stonebraker

Amol Deshpande

MITUniversity

of Maryland

Supervisors

collaborate

work sabbatical

Optimizations?

GraphiQL

Alekh Jindal

Sam Madden

Mike Stonebraker

Amol Deshpande

MITUniversity

of Maryland

Supervisors

collaborate

work sabbatical

Alekh Jindal

Sam Madden

Mike Stonebraker

Amol Deshpande

MITUniversity

of Maryland

Supervisors

collaborate

work sabbatical

Alekh Jindal

Sam Madden

Mike Stonebraker

Amol Deshpande

MITUniversity

of Maryland

Supervisors

collaborate

work sabbatical

Alekh Jindal

Sam Madden

Mike Stonebraker

Amol Deshpande

MITUniversity

of Maryland

Supervisors

collaborate

work sabbatical GraphiQL

Key Features• Graph view of relational data; the system takes

care of mapping to the relational world

• Inspired from PigLatin: right balance between declarative and procedural style language

• Key graph constructs: looping, recursion, neighborhood access

• Compiles to optimized SQL

Graph Table Relational Table

Alekh Jindal

Sam Madden

Mike Stonebraker

Amol Deshpande

MITUniversity

of Maryland

Supervisors

collaborate

work sabbatical

GraphiQL SQL

Graph Table

Graph Elementsnode1node2edge1edge2edge3node3node4node5

weight typeid

outgoingincoming

Graph Table

Graph Table Definition

• Create

• Load

• Drop

CREATE GRAPHTABLE g AS NODE (p1,p2,..) EDGE (q1,q2,..)

LOAD g AS NODE FROM graph_nodes DELIMITER d EDGE FROM graph_edges DELIMITER d

DROP GRAPHTABLE g

Graph Table Manipulation

• Iterate

• Filter

• Retrieve

• Update

• Aggregate

FOREACH element in g [WHILE condition]

g’ = g(k1=v1,k2=v2,…,kn=vn)

GET expr1,expr2,…,exprn [WHERE condition]

SET variable TO expr [WHERE condition]

SUM, COUNT, MIN, MAX, AVG

Nested Manipulation inner outer Iterate Aggregate Retrieve Update

Iterate

Aggregate

Retrieve

Update

Example 1: PageRank

FOREACH n IN g(type=N) SET n.pr TO new_pr

Example 1: PageRank

FOREACH n IN g(type=N) SET n.pr TO 0.15/num_nodes + 0.85*SUM(pr_neighbors)

Example 1: PageRank

FOREACH n IN g(type=N) SET n.pr TO 0.15/num_nodes + 0.85*SUM( FOREACH n’ IN n.in(type=N) GET pr_n’ )

Example 1: PageRank

FOREACH n IN g(type=N) SET n.pr TO 0.15/num_nodes + 0.85*SUM( FOREACH n’ IN n.in(type=N) GET n’.pr/COUNT(n’.out(type=N)) )

Example 1: PageRank

FOREACH iterations IN [1:10] FOREACH n IN g(type=N) SET n.pr TO 0.15/num_nodes + 0.85*SUM( FOREACH n’ IN n.in(type=N) GET n’.pr/COUNT(n’.out(type=N)) )

Example 1: PageRank

FOREACH iterations IN [1:10] FOREACH n IN g(type=N) SET n.pr TO 0.15/num_nodes + 0.85*SUM( FOREACH n’ IN n.in(type=N) GET n’.pr/COUNT(n’.out(type=N)) )

Reason about graphNeighborhood Access

Looping

Nested Manipulations

Example 2: SSSP

FOREACH n IN g(type=N) SET n.dist TO min_dist

Example 2: SSSP

FOREACH n IN g(type=N) SET n.dist TO MIN(n.in(type=N).dist)+1 AS dist’ WHERE dist’ < n.dist

Example 2: SSSP

WHILE updates > 0 FOREACH n IN g(type=N) updates = SET n.dist TO MIN(n.in(type=N).dist)+1 AS dist’ WHERE dist’ < n.dist

Example 2: SSSP

SET g(type=N).dist TO inf SET g(type=N,id=start).dist TO 0 WHILE updates > 0 FOREACH n IN g(type=N) updates = SET n.dist TO MIN(n.in(type=N).dist)+1 AS dist’ WHERE dist’ < n.dist

GraphiQL Compiler• Graph Table manipulations to relational

operators:- filter � selection predicates - iterate � driver loop - retrieve � projections - update � update in place - aggregate � group-by aggregate

• Graph Tables to relational tables:- mapping

GraphiQL Compilerg(type=N) � N

g(type=E) � E

g(type=N).out(type=E) � N ⋈ E

g(type=E).out(type=E) � E ⋈ E

g(type=N).out(type=N) � N ⋈ E ⋈ N

g.out.in = g.in

g.in.out = g.out

Example: SSSP

SET g(type=N).dist TO inf SET g(type=N,id=start).dist TO 0 WHILE updates > 0 FOREACH n IN g(type=N) updates = SET n.dist TO MIN(n.in(type=N).dist)+1 AS dist’ WHERE dist’ < n.dist

lupdateCount>0 ( n.dist ← σn.dist>dist’ ( !min(n’.dist)+1 ( "n.id ( N ⋈ E ⋈ N’ ) ) ) )

GraphiQL Optimizations

• De-duplicating graph elements

• Selection pushdown

• Cross-product as join

• Pruning redundant joins

Performance

Machine:

2GHz, 24 threads, 48GB memory, 1.4TB disk

Performance

Machine:

2GHz, 24 threads, 48GB memory, 1.4TB disk

Dataset:

Small: 81k/1.7m directed; 334k/925k undirected Large: 4.8m/68m directed; 4m/34m undirected

Performance - small graphTi

PageRank

Shortest Path

Triangles (global)

Triangles (local)

Strong Overlap

Weak Ties

Apache GiraphGraphiQL 12x Speedup!

PageRank

Shortest Path

Triangles (global)

Triangles (local)

Strong Overlap

Weak Ties

Apache GiraphGraphiQL

Performance - large graph

4.3x Speedup!

Summary• Several real world graph analysis are better off in

relational databases

• We need both the graph as well as relational view of data

• GraphiQL introduces Graph Tables to allows users to think in terms of graphs

• Graph Table supports recursive association, nested manipulations, and SQL compilation

• GraphiQL allows users to easily write a variety of graph analysis

Thanks!

Other LanguagesImperative languages: e.g. Green Marl XPath: e.g. Cypher, Gremlin Datalog: e.g. Socialite SPARQL: Teradata blog Procedural language: e.g. Vertex-centric

Graph Intuitive Query Language for Relational …jindal-web.appspot.com/slides/IEEEBigData.pdfIEEE...

Documents

Quality in Phrase Mining - Massachusetts Institute of ...people.csail.mit.edu/alekh/papers/Thesis.pdf · Quality in Phrase Mining ... The strategies aim to prune the candidate set

Selecting Subexpressions to Materialize at Datacenter Scale · 2020. 10. 9. · Selecting Subexpressions to Materialize at Datacenter Scale Alekh Jindal Konstantinos Karanasos Sriram

Vowpal Wabbit 6 - raw.githubusercontent.com · Machine Learning. See Miro & Alekh parts. VW is (by far) the most scalable public linear learner, and plausibly the most scalable anywhere

On the Theory of Policy Gradient Methods: Optimality, … · 2020-07-07 · On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shift Alekh Agarwal∗

artscapes 2020 selected artists · 2020-01-27 · Amisha Gupta Mohali Mohali Aparna Alekh kudtarkar Anandatman Mumbai Maharashtra Apurva Vinod Bhande The Cociety Dombivli Maharashtra

Schlumberger - CTech Lab€¦ · Barmer Dry Sanitation Facility at Laliyon ki Dhani Govt. School, Barmer ! Inaugurated in 9th August 2016 by Alekh Gupta, FMS WPS, Principal Sir

CLASSIFIED EMPLOYEE WORK CALENDAR 2019 - 2020 · Work Days Work Days Work Days Work Days Work Days Work Days Work Days Work Days

Taming the monster: A fast and simple algorithm for contextual bandits PRESENTED BY Satyen Kale Joint work with Alekh Agarwal, Daniel Hsu, John Langford,

MAP-REDUCE - Peoplepeople.csail.mit.edu/alekh/slides/mpi_retreat09.pdf · 2015. 2. 2. · Map(): Apply function f to each item of input list of key-value pairs and emit intermediate

Computation Reuse in Analytics Job Service at Microsoftjindal-web.appspot.com/posters/cloudviews-poster-sigmod18.pdf · Computation Reuse in Analytics Job Service at Microsoft Alekh

· Detail WORK WORK planner WORK WORK WORK WORK coder designer

Corralling a Band of Bandit Algorithms · 2020. 1. 3. · Corralling a Band of Bandit Algorithms Alekh Agarwal1, Haipeng Luo1, Behnam Neyshabur2, Rob Schapire1 1Microsoft Research,

BigDansing: A System for Big Data Cleansingilyas/papers/ZuhairSIGMOD2015.pdf · BigDansing: A System for Big Data Cleansing Zuhair Khayyaty Ihab F. Ilyasz Alekh Jindal] Samuel Madden]

7 Things to Know - MIT CSAILpeople.csail.mit.edu/alekh/posters/socc11.pdf · serve that for those queries referencing few attributes, e.g. Q 4 in LineItem and all queries in LineOrder,

WORK WORK WORK

Peregrine: Workload Optimization for Cloud Query Enginesaroy/peregrine-socc19.pdf · Peregrine: Workload Optimization for Cloud Query Engines Alekh Jindal, Hiren Patel, Abhishek Roy,

OctopusDB - MITpeople.csail.mit.edu/alekh/slides/QE2010.pdfMay 31, 2010 Alekh Jindal Information Systems Group Posting Lists Example: Banking 3 OLTP OLAP Streaming System Archival

AutoToken: Predicting Peak Parallelism for Big Data Analytics at … · 2020. 10. 9. · AutoToken: Predicting Peak Parallelism for Big Data Analytics at Microsoft Rathijit Sen Alekh

arXiv:1410.0440v1 [cs.LG] 2 Oct 2014djhsu/papers/apple-arxiv.pdf · Scalable Nonlinear Learning with Adaptive Polynomial Expansions Alekh Agarwal 1, Alina Beygelzimer2, Daniel Hsu3,

WWHow! Freeing Data Storage from Cagespeople.csail.mit.edu/alekh/papers/JQD13.pdf · UC 2: RAID master copy of data, multiple storage devices unchanged data layouts one or more replicas