Embed Size (px)
Citation preview
K-D Trees
CSE 373Data Structures
Lecture 22
12/6/02 K-D Trees - Lecture 22 2
Geometric Data Structures
• Organization of points, lines, planes, … to support faster processing
• Applications– Astrophysical simulation – evolution of galaxies– Graphics – computing object intersections– Data compression
• Points are representatives of 2x2 blocks in an image
• Nearest neighbor search
12/6/02 K-D Trees - Lecture 22 3
k-d Trees
• Jon Bentley, 1975• Tree used to store spatial data.
– Nearest neighbor search.– Range queries.– Fast look-up
• k-d tree are guaranteed log2 n depth where n is the number of points in the set.– Traditionally, k-d trees store points in d-
dimensional space which are equivalent to vectors in d-dimensional space.
12/6/02 K-D Trees - Lecture 22 4
Range Queries
x
y
ab
f
c
gh
ed
i
x
y
ab
f
c
gh
ed
i
Rectangular query Circular query
12/6/02 K-D Trees - Lecture 22 5
x
y
ab
f
c
gh
ed
i
Nearest Neighbor Search
query
Nearest neighbor is e.
12/6/02 K-D Trees - Lecture 22 6
k-d Tree Construction
• If there is just one point, form a leaf with that point.• Otherwise, divide the points in half by a line
perpendicular to one of the axes. • Recursively construct k-d trees for the two sets of
points.• Division strategies
– divide points perpendicular to the axis with widest spread.– divide in a round-robin fashion (book does it this way)
12/6/02 K-D Trees - Lecture 22 7
x
y
k-d Tree Construction (1)
ab
f
c
gh
ed
i
divide perpendicular to the widest spread.
12/6/02 K-D Trees - Lecture 22 8
y
k-d Tree Construction (2)
x
ab
c
gh
ed
i s1
s1
x
f
12/6/02 K-D Trees - Lecture 22 9
y
k-d Tree Construction (3)
x
ab
c
gh
ed
i s1
s2y
s1
s2
x
f
12/6/02 K-D Trees - Lecture 22 10
y
k-d Tree Construction (4)
x
ab
c
gh
ed
i s1
s2y
s3x
s1
s2
s3
x
f
12/6/02 K-D Trees - Lecture 22 11
y
k-d Tree Construction (5)
x
ab
c
gh
ed
i s1
s2y
s3x
s1
s2
s3
a
x
f
12/6/02 K-D Trees - Lecture 22 12
y
k-d Tree Construction (6)
x
ab
c
gh
ed
i s1
s2y
s3x
s1
s2
s3
a b
x
f
12/6/02 K-D Trees - Lecture 22 13
y
k-d Tree Construction (7)
x
ab
c
gh
ed
i s1
s2y
s3x
s4y
s1
s2
s3
s4
a b
x
f
12/6/02 K-D Trees - Lecture 22 14
y
k-d Tree Construction (8)
x
ab
c
gh
ed
i s1
s2y
s3x
s4y
s5x
s1
s2
s3
s4
s5
a b
x
f
12/6/02 K-D Trees - Lecture 22 15
y
k-d Tree Construction (9)
x
ab
c
gh
ed
i s1
s2y
s3x
s4y
s5x
s1
s2
s3
s4
s5
a b
dx
f
12/6/02 K-D Trees - Lecture 22 16
y
k-d Tree Construction (10)
x
ab
c
gh
ed
i s1
s2y
s3x
s4y
s5x
s1
s2
s3
s4
s5
a b
d ex
f
12/6/02 K-D Trees - Lecture 22 17
y
k-d Tree Construction (11)
x
ab
c
gh
ed
i s1
s2y
s3x
s4y
s5x
s1
s2
s3
s4
s5
a b
d e
g
x
f
12/6/02 K-D Trees - Lecture 22 18
y
k-d Tree Construction (12)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s5x
s1
s2
s3
s4
s5
s6
a b
d e
g
x
f
12/6/02 K-D Trees - Lecture 22 19
y
k-d Tree Construction (13)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s5x
s1
s2
s3
s4
s5
s6
s7a b
d e
g
x
f
12/6/02 K-D Trees - Lecture 22 20
y
k-d Tree Construction (14)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s5x
s1
s2
s3
s4
s5
s6
s7a b
d e
g c
x
f
12/6/02 K-D Trees - Lecture 22 21
y
k-d Tree Construction (15)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s5x
s1
s2
s3
s4
s5
s6
s7a b
d e
g c f
x
f
12/6/02 K-D Trees - Lecture 22 22
y
k-d Tree Construction (16)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d e
g c f
x
f
12/6/02 K-D Trees - Lecture 22 23
y
k-d Tree Construction (17)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d e
g c f h
x
f
12/6/02 K-D Trees - Lecture 22 24
y
k-d Tree Construction (18)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d e
g c f h i
x
f
k-d tree cell
12/6/02 K-D Trees - Lecture 22 25
2-d Tree Decomposition
1
2
3
12/6/02 K-D Trees - Lecture 22 26
x
y
k-d Tree Splitting
ab
c
gh
ed
i a d g b e i c h fa c b d f e h g i
xy
sorted points in each dimension
f
• max spread is the max offx -ax and iy - ay.
• In the selected dimension themiddle point in the list splits the data.
• To build the sorted lists for the other dimensions scan the sortedlist adding each point to one of two sorted lists.
1 2 3 4 5 6 7 8 9
12/6/02 K-D Trees - Lecture 22 27
x
y
k-d Tree Splitting
ab
c
gh
ed
i a d g b e i c h fa c b d f e h g i
xy
sorted points in each dimension
f
1 2 3 4 5 6 7 8 9
a b c d e f g h i
0 0 1 0 0 1 0 1 1
indicator for each set
scan sorted points in y dimensionand add to correct set
a b d e g c f h iy
12/6/02 K-D Trees - Lecture 22 28
k-d Tree Construction Complexity
• First sort the points in each dimension.– O(dn log n) time and dn storage.– These are stored in A[1..d,1..n]
• Finding the widest spread and equally divide into two subsets can be done in O(dn) time.
• We have the recurrence– T(n,d) < 2T(n/2,d) + O(dn)
• Constructing the k-d tree can be done in O(dn log n) and dn storage
12/6/02 K-D Trees - Lecture 22 29
Node Structure for k-d Trees
• A node has 5 fields– axis (splitting axis)– value (splitting value)– left (left subtree)– right (right subtree)– point (holds a point if left and right children are null)
12/6/02 K-D Trees - Lecture 22 30
k-d Tree Nearest Neighbor Search
• Search recursively to find the point in the same cell as the query.
• On the return search each subtree where a closer point than the one you already know about might be found.
12/6/02 K-D Trees - Lecture 22 31
y
k-d Tree NNS (1)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d e
g c f h i
x
f
query point
12/6/02 K-D Trees - Lecture 22 32
y
k-d Tree NNS (2)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d e
g c f h i
x
f
query point
12/6/02 K-D Trees - Lecture 22 33
y
k-d Tree NNS (3)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d e
g c f h i
x
f
query point
12/6/02 K-D Trees - Lecture 22 34
y
k-d Tree NNS (4)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d e
g c f h i
x
f
query point
w
12/6/02 K-D Trees - Lecture 22 35
y
k-d Tree NNS (5)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d e
c f h i
x
f
query point
w
g
12/6/02 K-D Trees - Lecture 22 36
y
k-d Tree NNS (6)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d e
c f h i
x
f
query point
w
g
12/6/02 K-D Trees - Lecture 22 37
y
k-d Tree NNS (7)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d e
c f h i
x
f
query point
w
g
12/6/02 K-D Trees - Lecture 22 38
y
k-d Tree NNS (8)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d e
c f h i
x
f
query point
w
g
12/6/02 K-D Trees - Lecture 22 39
e
y
k-d Tree NNS (9)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d
c f h i
x
f
query point
w
g
12/6/02 K-D Trees - Lecture 22 40
y
k-d Tree NNS (10)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d
c f h i
x
f
query point
w
e
g
12/6/02 K-D Trees - Lecture 22 41
y
k-d Tree NNS (11)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d
c f h i
x
f
query point
w
e
g
12/6/02 K-D Trees - Lecture 22 42
y
k-d Tree NNS (12)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d
c f h i
x
f
query point
w
e
g
12/6/02 K-D Trees - Lecture 22 43
y
k-d Tree NNS (13)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d
c f h i
x
f
query point
w
e
g
12/6/02 K-D Trees - Lecture 22 44
y
k-d Tree NNS (14)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d
c f h i
x
f
query point
w
e
g
12/6/02 K-D Trees - Lecture 22 45
y
k-d Tree NNS (15)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d
c f h i
x
f
query point
w
e
g
12/6/02 K-D Trees - Lecture 22 46
y
k-d Tree NNS (16)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d
c f h i
x
f
query point
w
e
g
12/6/02 K-D Trees - Lecture 22 47
y
k-d Tree NNS (17)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d
c f h i
x
f
query point
w
e
g
12/6/02 K-D Trees - Lecture 22 48
y
k-d Tree NNS (18)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d
c f h i
x
f
query point
w
e
g
12/6/02 K-D Trees - Lecture 22 49
y
k-d Tree NNS (19)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d
c f h i
x
f
query point
w
e
g
12/6/02 K-D Trees - Lecture 22 50
y
k-d Tree NNS (20)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d
c f h i
x
f
query point
w
e
g
12/6/02 K-D Trees - Lecture 22 51
y
k-d Tree NNS (21)
x
ab
c
gh
ed
i s1
s2y y
s6
s3x
s4y
s7y
s8y
s5x
s1
s2
s3
s4
s5
s6
s7
s8
a b
d
c f h i
x
f
query point
w
e
g
12/6/02 K-D Trees - Lecture 22 52
Nearest Neighbor SearchNNS(q: point, n: node, p: point, w: distance) : point {if n.left = null then {leaf case}
if distance(q,n.point) < w then return n.point else return p;else
if w = infinity thenif q(n.axis) < n.value then
p := NNS(q,n.left,p,w);w := distance(p,q);if q(n.axis) + w > n.value then p := NNS(q, n.right, p, w);
elsep := NNS(q,n.right,p,w);w := distance(p,q);if q(n.axis) - w < n.value then p := NNS(q, n.left, p, w);
else //w is finite//if q(n.axis) - w < n.value then p := NNS(q, n.left, p, w);w := distance(p,q);if q(n.axis) + w > n.value then p := NNS(q, n.right, p, w);
return p}
Main is NNS(q,root,null,infinity)
12/6/02 K-D Trees - Lecture 22 53
The Conditionalq(n.axis) + w > n.value
n.value
n.axis = x
q
w
q(n.axis) + w
Current nearest point
12/6/02 K-D Trees - Lecture 22 54
Notes on k-d NNS
• Has been shown to run in O(log n) average time per search in a reasonable model. (Assume d a constant)
• Storage for the k-d tree is O(n).• Preprocessing time is O(n log n) assuming d
is a constant.
12/6/02 K-D Trees - Lecture 22 55
Geometric Data Structures
• Geometric data structures are common.• The k-d tree is one of the simplest.
– Nearest neighbor search– Range queries
• Other data structures used for– 3-d graphics models– Physical simulations
