1

On Efficient Matching of Streaming XML Documents and Queries

Laks V.S. Lakshmanan1

P. Sailaja2

1 University of British Columbia, Canada

2 Indian Inst. of Tech., Bombay, India (work performed while visiting IIT-Bombay).

UBC, Canada EDBT 2002, Prague. 2

Outline

I. Motivating Applications

II. Problem

III. Dual Index

IV. Algorithms

V. Experiments

VI. Summary & Future Work

UBC, Canada EDBT 2002, Prague. 3

Motivating Application 1

Information dissemination in the large Numerous data sources on the web

Traditional means: search and browse

Alternative – publish and subscribe

System matches (new) data to subscribers’

interests

Periodic notification

UBC, Canada EDBT 2002, Prague. 4

Motivating Application 2

Supply chain automation Catalog of products and services from

suppliers (data) Registered sets of requirements

(subscriptions) from manufacturing units Notify relevant consumers upon arrival

of new data Other applications include electronic

auctioning, online shopping, etc.

UBC, Canada EDBT 2002, Prague. 5

Problem

Matching specifications (of products, services, etc.) to requirements (subscriptions) efficiently.

Specs – akin to data. Requirements – queries. Data may stream through. Quickly determine which

subscribers/users a piece of data is relevant to.

UBC, Canada EDBT 2002, Prague. 6

Problem Traditional setting:

Large DB One (at most a few) query at a time

Our problem: A small DB (a tuple, XML doc, etc.) Large no. of queries Dual to traditional problem

Focus of this paper: data = XML docs Queries = a fragment of XPath

UBC, Canada EDBT 2002, Prague. 7

Problem (Formalized) Given

an XML document a large number of XPath queries

Determine which queries are answered by each element (formalized using matching)

Query labeling: label each node with sets of queries answered by the subtree rooted there

Naïve Approach doesn’t scale w/ no. of queries Main challenge: small (1 or 2) # passes over

data tree

UBC, Canada EDBT 2002, Prague. 8

An Exampe Query

<Result> FOR $p IN

document(“catalog.xml”)//part, $b in $p/brand, $q IN $p//part WHERE A2D IN $q/name AND AMD IN $q/brand RETURN $p </Result>

UBC, Canada EDBT 2002, Prague. 9

Problem (An Example)

UBC, Canada EDBT 2002, Prague. 10

Dual Index Traditional index – quickly localize

search for data matching query pattern

Dual index – for each primitive pattern,

determine (sub)queries to which they

are relevant

Choice of primitive patterns depends on

type of data (e.g., XML vs. relational)

And on classes of queries considered

(e.g., chains vs. trees)

UBC, Canada EDBT 2002, Prague. 11

Tree Dual Index

Primitive “access path” questions to be answered: For a constant c, what are leaf appearances? For a tag t, what are non-leaf appearances? What query

nodes are its pc- and ad-children? Example:

a 1

b 2 c 3

a 4

b 5 c 6

b 1

c 2 a 6

c 3

b 5a 4

P Q

Index entry for a:

DI(a)[L]: (P, 3, {}), (Q, 6, {4,6})* *

DI(a)[N]: (P, 1, F, {2,3}, {}),

(P, 4, T, {6}, {5}).

UBC, Canada EDBT 2002, Prague. 12

Tree Labeling Algorithm – 3 Lists

a 1

b 2 c 9

c 3 a 8 d 10

c 4 a 11 b 15

a 5 d 6 c 12 d 13

b 7 b 14

3 lists (conceptually)

TML(u): (Query, query node, DN, ans-node)

PL(u): (P,l,m,x): rel

QL(u): Query Ids

UBC, Canada EDBT 2002, Prague. 13

Tree Labeling Algo. – TML base case

a 1

b 2 c 9

c 3 a 8 d 10

c 4 a 11 b 15

a 5 d 6 c 12 d 13

b 7 b 14

(P,m,{v1, …, vk}) DI(t)[L]

(P,v1,m,?), …, (P,vk,m,?) TML(u),

whenever u.tag= t;

e.g.: DI(a)[L] has (Q,6,{4,6}).

So, add (Q,4,6,?), & (Q,6,6,?) to TML()

(Q,6,6,?) (Q,6,6,i), i = 1,5, 8, 11.

If vi=m, ? u.

UBC, Canada EDBT 2002, Prague. 14

Tree Labeling Algo. – TML PL

a 1

b 2 c 9

c 3 a 8 d 10

c 4 a 11 b 15

a 5 d 6 c 12 d 13

b 7 b 14

(P,l,m,x) TML(u)

(P,l,m,x):child PL(parent(u)).

(P,l,m,x):desc PL(anc(u)).

e.g.: (Q,4,6,?) PL(5)

So, (Q,4,6,?):child PL(4).

And (Q,4,6,?):desc PL(i), i= 3, 2, 1.

Optimizations possible, but suppressed.

UBC, Canada EDBT 2002, Prague. 15

Tree Labeling Algo. – TML inductive case

a 1

b 2 c 9

c 3 a 8 d 10

c 4 a 11 b 15

a 5 d 6 c 12 d 13

b 7 b 14

(P,l,B,C,D) DI(t)[N]

c C: (P,c,m,y):child PL(u) &

d D: (P,d,m,y):rel PL(u)

(P,l,m,x) TML(u).

If l=m, x u.

e.g.: (P,4,T,{6},{5}) DI(a)[N].

(P,6,3,?) TML(12), so (P,6,3,?):child PL(11).Similarly, (P,5,3,?):desc PL(11)

So, (P,4,3,?) TML(11).

UBC, Canada EDBT 2002, Prague. 16

Tree Labeling Algo. – QL

a 1

b 2 c 9

c 3 a 8 d 10

c 4 a 11 b 15

a 5 d 6 c 12 d 13

b 7 b 14

• TML, PL, feed each other.

•QL – special case of TML

•P QL(u) iff

(P,1,m,x) TML(x).

•e.g.: (P,1,3,9) TML(1),

so P QL(9).

& (Q,1,6,5) TML(2), so

Q QL(5).

UBC, Canada EDBT 2002, Prague. 17

Tree Labeling – Summary

labeling completed in two passes pass 1: compute TML/PL (bottom-up) pass 2: compute QL (top-down)

no. of I/O invocations is 2 * # data tree nodes.

Other algorithms in paper: chain labeling chain split labeling of trees

UBC, Canada EDBT 2002, Prague. 18

Experiments

matchMaker implementation: JDK1.3 and C++ storage – BerkeleyDB 3.17 dual index stored in disk lists manipulated in memory Intel PIII, 1GB RAM, 512K cache, Linux 7.0

Data sets: generated using IBM’s XML Gen tool conforming to GEDCOM DTD (geological

data) (about 120 elements)

UBC, Canada EDBT 2002, Prague. 19

Experiments document depth 10; avg fanout – [2, 5] chain labeling algorithm is at least 5 times

faster than query-at-a-time approach For tree labeling, query-at-a-time doesn’t

produce results in reasonable time! Focus of experiments (for trees):

Direct tree labeling algorithm vs. chain split algorithm (not discussed)

UBC, Canada EDBT 2002, Prague. 20

Experiments

UBC, Canada EDBT 2002, Prague. 21

Experiments

UBC, Canada EDBT 2002, Prague. 22

Experiments

UBC, Canada EDBT 2002, Prague. 23

Related Work Documents – user profile match (IR) Notion of standing queries – long history:

E.g., Tapesty, TriggerMan, NiagaraCQ, etc. Publish-and-subscribe – Fabret et al. 00,

01. Patterns: boolean combo of relOp comp value

XFilter 00, 01. Only determine if doc contains an answer Multiple answers in one doc not considered

UBC, Canada EDBT 2002, Prague. 24

Related Work XTrie approach

Decompose query tree into ad-free chains Index using trie Determine only if a doc contains an answer

Main distinguishing features of matchMaker: Answers located Multiple answers per doc All proposed algorithms – guaranteed

resource bounds (e.g., #passes, I/O)

UBC, Canada EDBT 2002, Prague. 25

Summary & Future Work

Matching large no. of queries to XML data trees (as they stream through)

Dual to usual query processing Dual index (chains vs. trees) Algorithms for query labeling of data

trees Making algorithms more efficient (single

pass algorithm for chains: done) Expanding classes of queries handled Algebra for this dual query processing

problem?