Retrieving k-Nearest Neighboring Trajectories by a Set of Point Locations

Lu-An Tang, Yu Zheng, Xing Xie, Jing Yuan, Xiao Yu, Jiawei Han

•University of Illinois at Urbana-Champaign

Microsoft Research Asia

2

Motivation: trajectory query by locations

Huge volume of spatial trajectories

Require to search trajectories by a set of point locations

Geo-tagged photos Taxi trajectories Check-ins

User

3

k-Nearest Neighboring trajectory query

Query the top k trajectories with the minimum aggregated distance to the given locations

The trajectories may not exactly pass those locations

q1

q2

q3

4

k-NNT query

Task Definition:

Given the trajectory dataset D, and a set of query points, Q, the k-NNT query

retrieves k trajectories K from D, K = {R1, R2, …, Rk} that for ∀ Ri ∈ K, ∀ Rj ∈ D -

K, dist(Ri,Q) ≤ dist(Rj,Q).

ChallengesHuge trajectory dataset: High I/O cost to scan all the trajectories

Aggregated distance computation

Non-uniform distribution:

the trajectories are sparse/dense in different regions

the user-given query locations may be far from all the trajectories

5

R1

R2q1

q2 q3

p1,1p1,2

p1,3p1,4 p1,5

p2,1p2,2

p2,3

p2,4

p2,5

p2,6

The aggregate distance in k-NNT query

1. Find out the closest point from a trajectory to each query point (i.e., shortest matching pairs)

3. Sum up the lengths of all matching pairs

• dist(R1, q1)= dist(p1,2, q1)= 20 m• dist(R1, q2)= dist(p1,3, q2)= 50 m• dist(R1, q3)= dist(p1,5, q3)= 15 m• dist(R1, Q)=∑ dist(R1, qi)= 85 m

• dist(R2, q1)= dist(p2,3, q1)= 30 m• dist(R2, q2)= dist(p2,4, q2)= 5 m• dist(R2, q3)= dist(p2,6, q3)= 40 m• dist(R2, Q)=∑ dist(R2, qi)= 75 m

6

Related Work: k-BCT query

k-Best Connected Trajectory (k-BCT) query [SIGMOD2010]the similarity function between a trajectory R and query locations Q is

Problem: This function changes over units (inconsistent)

An example If query Q has two points q1 and q2;

dist(R1, q1) = dist(R1, q2) = 2.4km = 1.48 miles,

dist(R2, q1) = 1.5 km =0.93 miles, dist(R2, q2) = 5km = 3.1 miles

Use unit “mile”, Sim(R1, Q) = 0.45 > Sim(R2, Q) = 0.43

Use unit “km”, Sim(R1, Q) = 0.18 < Sim(R2, Q) = 0.22

7

Advantages of k-NNT over k-BCT

The distance function of k-BCT changes over units (inconsistent)

The distance function of k-BCT is sensitive to a query

q1

q2

q3

• k-BCT&k-NNT

• k-NNT

• k-BCT

8

Query framework: candidate-generation-and-verification

Candidate generation

Best-first search based individual heaps

Coordination by a global heap

Candidate verification

Lower-bound estimation

Efficient pruning with the global heap

Outlier query location

Qualifier expectation based method

R1 R2R3 R4

q1

q2

q3

R5

R6

dist(R1, Q)= 5+2+2=9 mdist(R2, Q)= 25+20+30=75mdist(R3, Q)= 80+25+30=135mdist(R4, Q)= 90+5+3=98 mdist(R5, Q)= 55+8+70=123mdist(R6, Q)= 120+80+40=240 m

Direct Computing

Candidate Generation

R1 R4

q1

q2

q3

R5 dist(R1, Q)= 5+2+2=9 mdist(R4, Q)= 90+5+3=98 mdist(R5, Q)= 55+8+70=123m

Candidate Verification

9

Candidate Generation

Given a query Q = {q1, q2, …, qm}, generate a trajectory

candidate set including all the k-NNTs (i.e., complete set)

Step 1: searching k-NN points using best-first-based individual heap

Step 2: generating the candidate trajectories by the global heap

R1 R2R3 R4

q1

q2

q3

R5

R6

<p2,3, q1><p5,2, q1><p1,6, q1><p2,9, q1>

…...

h1

<p6,2, q2><p5,3, q2><p7,4, q2><p4,8, q2>

…...

h2

<p2,2, q3><p3,5, q3><p7,3, q3><p8,6, q3>

…...

h3

10

Global heap

A minimum heap sorting matching pairs by the distance

Retrieves new matching pair from individual heaps

Pops the matching pairs to the candidate set

Step 2: generating candidate trajectories

<p2,3, q1><p5,2, q1><p1,6, q1><p2,9, q1>

…...

h1

<p6,2, q2><p5,3, q2><p7,4, q2><p4,8, q2>

…...

<p2,2, q3><p3,5, q3><p7,3, q3><p8,6, q3>

…...

<p5,1, qm><p2,3, qm><p5,7, qm><p9,2, qm>

…...

…...

<p1,4, q1>, <p5,1, q3>, <p6,4, q4>, <p3,4, q2>, …...

Global Heap (Size=m)

R1: <p1,2, q1>, <p1,5, q2>, <p1,3, q3>, ……, <p1,9, qm>. R2: , <p2,2, q2>, <p2,4, q3>, ……, . R4: <p4,5, q1>, , <p4,3, q3>, ……, <p4,7, qm> ………... Candidate Set

h2 h3 hmIndividual Heaps

Advantages

guarantee including all k-NNTs in candidate set

generate compact candidate sets

11

R1R2

R3 R4

R5

p1,2

p4,4

p4,5p1,4

p1,6

p5,5

Example: Search based on the global heap

Candidate Set

Global Heap

Individual Heaps

q1

q2

q3

h1 h2 h3

…… …… ……

• <p1,2, q1>

• <p1,4, q2>

• <p1,6, q3>

12

R1R2

R3 R4

R5

p1,2

p4,4

p4,5p1,4

p1,6

p5,5

Example: Search based on the global heap

Candidate Set

Global Heap

Individual Heaps

q1

q2

q3

h1 h2 h3

…… …… ……

• <p1,2, q1>

• <p1,4, q2>

• <p1,6, q3>

• R1: (Partial Match)

• <p5,5, q2>

13

R1R2

R3 R4

R5

p1,2

p4,4

p4,5p1,4

p1,6

p5,5

Example: Search based on the global heap

Candidate Set

Global Heap

Individual Heaps

q1

q2

q3

h1 h2 h3

…… …… ……

• <p1,2, q1>

• <p1,4, q2>

• <p1,6, q3>

• R1: (Partial Match)

• <p5,5, q2>

• <p4,5, q3>

14

R1R2

R3 R4

R5

p1,2

p4,4

p4,5p1,4

p1,6

p5,5

Example: Search based on the global heap

Candidate Set

Global Heap

Individual Heaps

q1

q2

q3

h1 h2 h3

…… …… ……

• <p1,2, q1>

• <p1,4, q2>

• <p1,6, q3>

• R1: (Partial Match)

• <p5,5, q2>

• <p4,5, q3>

• R5: (Partial Match)

• <p4,4, q2>

15

Example: Search based on the global heap

R1: <p1,2, q1>, <p1,4, q2>, <p1,6, q3>. (Full Match)

R4: <p4,5, q3>. (Partial Match)

R5: <p5,5, q2>. (Partial Match)

Candidate Set

Global Heap<p1,2, q1>, <p4,4, q2>, <p1,5, q3>

Individual Heaps

…… ……

h1 h2 h3

……

R1R2

R3 R4

R5

p1,2

p4,4

p4,5p1,4

p1,6

p5,5

q1

q2

q3

Stop critiria: when there is k full-matching candidates – Property 1: The candidate set is complete if G has popped out k full-matching candidates (In this example k=1)

16

Candidate verification

The full-matching candidate may not be the final k-NNT

The system has to retrieve the partial-matching trajectories (R4 and R5) to compute their aggregate distance (I/O cost)

Question: can we compute a lower-bound for R4 and R5

without retrieving their details?

If LB(R4/5) > dist(R1,Q), we can prune it directly

R1: <p1,2, q1>, <p1,4, q2>, <p1,6, q3>. (Full Match)

R4: <p4,5, q3>. (Partial Match)

R5: <p5,5, q2>. (Partial Match)

Candidate Set

17

Candidate verification

The lower-bound of a partial-matching trajectory is

If the LB(R) is larger than the distance of full-matching candidate, R can be pruned directly

R1: <p1,2, q1> <p1,4, q2> <p1,6, q3> dist(R1) = 95

R4: <p4,5, q3>

R5: <p5,5, q2>

Candidate Set

Global Heap• <p1,5, q3>

• <p1,2, q1>

• <p4,4, q2>

• <p1,5, q3>

• <p1,2, q1>

• <p4,4, q2>

• <p1,5, q3>

• <p1,2, q1>

• <p4,4, q2>

LB(R4) =114 (pruned)

LB(R5) =90 (passed)

18

Problem of Outlier Query Location

A query location is an outlier if it is far from all the trajectories

Too many partial-matching candidates will be generated before finding a full-matching candidates

R1: <p1,1, q1>, <p1,4, q2>, . (Partial Matching) R2: <p2,1, q1>, <p2,5, q2>, . (Partial Matching)

R4: , <p4,4, q2>, . (Partial Matching)

<p1,1, q1>, <p4,4, q2>, <p1,7, q3> Iteration 4

Global Heap

Candidate Set

<p1,4, q2>, <p1,1, q1>, <p1,7, q3> Iteration 3

<p2,1, q1>, <p1,4, q2>, <p1,7, q3> Iteration 2

<p2,5, q2>, <p2,1, q1>, <p1,7, q3> Iteration 1

……

<p1,7, q3> cannot be popped out

R1R2

R3 R4

q1

q2

q3

p1,2

p2,1

p2,2

p2,5

p1,7p2,6

p4,4

p1,4

19

Qualifier expectation based method

The system can make up the missing pairs of a partial-matching trajectory by retrieving all its points

Two key issues:Guarantee the completeness of candidate set

Property 2: If there are k made-up candidates (qualifier) with distance smaller than the sum of the pairs in global heap, the candidate set is complete

Which candidate should be selected to make up? The qualifier expectation measure

20

R1R2

R3 R4

q1

q2

q3

p1,2

p2,1

p2,2

p2,5

p1,7p2,6

p4,4

p1,4

Example of Qualifier Expectation

R1: <p1,1, q1>, <p1,4, q2>, .

R2: <p2,1, q1>, <p2,5, q2>, .

R4: ,<p4,4, q2>, .

Candidate Set

Global Heap, total dist sum(G) = 200m<p2,1, q1>, <p4,4, q2>, <p1,7, q3>

R1: 40m. R2: 30m. R4: 15m.

Qualifier Expectation

• R1: <p1,1, q1>, <p1,4, q2>, <p1,7, q3>.

dist(R1) =160m < sum(G), R1 is a qualifier

21

Experiment Setup

Real Dataset: collected from the Microsoft Geolife and T-Drive projects , with over 20,000 real trajectories

Synthetic datasets with both uniform distribution and biased distribution

Random generated query Q

The proposed methods are compared with Fagin’s Algorithm (FA) and Threshold Algorithm (TA) (used in k-BCT)

22

Evaluations on synthetic dataset (biased distribution)

GH (global heap) is faster than baselines with less I/O costs

QE( global heap+ qualifier expectation ) is an order of magnitude faster than others

1000

10000

100000

1000000

10000000

2 4 6 8 10

100

1000

10000

100000

2 4 6 8 10

100

1000

10000

100000

3k 6k 9k 12k

GH QE TA FA

Time (unit: ms) Accessed Rtree Nodes

(a) Query Time vs. |Q| (b) I/O Cost vs. |Q|

23

Evaluations on real dataset

When |Q| is small, the probability of outlier location is low, GH achieves the best performance

When |Q| is larger, the probability of outlier location is high, QE is more efficient

1000

10000

100000

1000000

2 4 6 8 10

10

100

1000

10000

2 4 6 8 10

100

1000

10000

100000

3k 6k 9k 12k

GH QE TA FA

Time (unit: ms) Accessed Rtree Nodes

(a) Query Time vs. |Q| (b) I/O Cost vs. |Q|

24

Conclusion

k-Nearest Neighboring Trajectory (k-NNT) queryretrieve trajectories by a set of locations

Candidate-generation-and-verification frameworkGenerate candidate trajectories with global heap

Efficient lower-bound computation

Outlier query location: qualifier expectation based method

25

Thanks very much!

Any Questions?