PROBING THE LOCAL-FEATURE SPACE OF INTEREST POINTS

Wei-Ting Lee, Hwann-Tzong Chen

Department of Computer ScienceNational Tsing Hua University, Taiwan

ICIP 2010

OUTLINE• Introduction• Approach– Locality-Sensitive

Hashing (LSH) – Sketching the

Feature Space• Experiments– Fast Matching

• Conclusion

INTRODUCTION

Local feature have been extensively used to represent image for various problem

Lots of local feature detector and local feature descriptor have been proposed recent years

Recent History

Maximally Stable Extremal Regions (MSER) [1] BMVC 2002

Difference-of-Gaussian and Scale-Invariant-Feature-Transform (SIFT) [2]

IJCV 2004

Affine invariant detector [3] , [4] IJCV 2004 , TPAMI 2005

Histogram of oriented gradients (HOG) [5]CVPR 2005

‘Visual words’ [6] ‘codebooks’ [7] ICCV 2003 , BMVC 2003

For example

• Present an empirical analysis of the feature space of interest points detected in natural image

• Perform an approximate method for the fast matching between two sets of interest points detected in two images

• Show that the complexity of matching M points to N points can be reduced from O(MN) to O(M+N)

INTRODUCTION

Locality-Sensitive Hashing

• p-stable Distribution:

Locality-Sensitive Hashing based on 2-Stable Distribution

Hash Family

a : random vector sampled from a Gaussian distribution

b : real value chosen uniformly from the range [0 , r]

r : line width

The dot-product a‧v projects each vector to the real line

Building Hash table

Building Hash table

Choose the width r based on the minimum and maximum

=？

θ

a‧b = |a| |b|

Index function

t = 5 , K=3

[5] [5] [5] = 125 = (5-1) * 52 + (5-1) * 51 + 4 * 50 + 1 = 4 * 25 + 20 + 4 + 1 = 125

Sketching the Feature Space

Berkeley segmentation database [14]

Use difference of Gaussian (DOG) [2] & Hessian-affine [3] detector detect about 200,000 interest points

Extract image patches by SIFT descriptor [2]

Create a hash table (L = 1) with five projection(K = 5) and 15 segments on each dot-product real line (t = 15)

The total number of buckets is 155 = 759,375

Entropy = 4.2251(a) DOG

Entropy = 4.0622(b) Hessian-affine

Sketching the Feature SpaceDistribution and Entropy

Collect three image patches of different size 16x16 , 32x32 , 64x64

Each set consist of 200,000 patches.

Natural image patches (from Berkeley segmentation database )

Noise image patches (Randomly-generated noise patches)

Sketching the Feature Space

Distribution and Entropy

Fast Matching

3

3

3 3

3 3

3

3

Referenceimage

RemainingImage (test)

Fast MatchingWe create L = 16 hash tables to probe the 128-dimensional SIFT-feature space

Each table is equipped with five 2-stable Projections , and the projected values are quantized into 15 segments,

i.e., K = 5 and t = 15

For LSH, we use two threshold values of dot-product, θ = 0.95 and θ = 0.97,to determine whether a pair of feature vectors in the same bucket yields a match

LSH is 2 to 15 times faster than matching by exhaustive search

a b = |a| |b| ‧If a = b , then = 1

Fast Matching

DoG detector + SIFT descriptor Hessian-affine detector + SIFT descriptor

DoG detector + SIFT descriptor

2-stable LSH matching vs. exhaustive matching

2-stable LSH matching vs. exhaustive matching

Hessian-affine detector + SIFT descriptor

Conclusion

Using the approximate nearest-neighbor probing scheme derived from 2-stable Locality-Sensitive Hashing

Make use of the efficient representation of the SIFT feature space, and present a fast feature-matching method for finding correspondences between two sets of interest points.

And,Have been used by Whiteorange !!

THANK YOU SO MUCH