Detection and Matching of Keypoints in Road

Scene Images

Shafeeq.Zhanpeng.James.Toan

Background & ObjectivesBackground:

1. Road scene images are taken in different views2. Road scene images contain moving objects3. Road scene images contain repeating objects

Objectives:

1. Keypoints are detected automatically from road scene.2. Automatic keypoints matching applied to any pair of

images with different viewpoints but of the same scene. 3. As robust as possible

MethodTrials

1- global context SIFT + Hungarian 2- SIFT + Hungarian 3- global context SIFT + Flann 4- SIFT + Flann 5- SURF + Hungarian 6- SURF + Flann

Current

Detection & Matching Key points: Affine-SIFT

Elimination of bad matches: Moisan-Stival Procedure + RANSAC

Affine-SIFTIn addition to SIFT, Affine-SIFT handles large transition tilt

Affine-SIFT is proven to be fully affine-invariant

Affine-SIFTSIFT is invariant only for 4 parameters: zoom, rotation and translation(x and y)

Affine-SIFT can also handle angles defining camera axis orientation

Affine-SIFT

Φ is the longitude angle between optical axis and a fixed vertical plane

Affine-SIFT

Θ is the latitude angle between optical axis and the normal to the image plane

Affine-SIFT algorithm

1. Apply rotation to both images

2. Apply simulated tilts to all rotated images

3. Perform SIFT of all pairs of resulting images

Affine-SIFT vs SIFT and SURF

Affine-SIFT produces more good key points

Affine-SIFT produces more matches

Affine-SIFT vs SIFTSIFT

Number of keypointsImage 1: 747Image 2: 704Number of matches: 200Number of correct matches after filtering bad matches: 171

Affine SIFT

Number of keypointsImage 1: 4010Image 2: 3909Number of matches: 687Number of correct matches after filtering bad matches: 687

Affine-SIFT vs SIFT

Affine-SIFT vs SURFSURF

Number of keypoints:Image 1: 727Image 2: 716Number of matches: 27Number of correct matches: 7

Affine SIFT:

KeypointsImage 1: 2480Image 2: 2692Number of matches: 23Number of good matches: 20

Affine-SIFT vs SIFTSIFT

Number of keypoints:Image 1: 377Image 2: 315Number of matches: 24Number of correct matches: 16

Affine SIFT:

KeypointsImage 1: 2428Image 2: 1662Number of matches: 26Number of good matches: 24

Moisan-Stival Procedure

Eliminate matches that are incoherent with epipolar geometry

Moissan-Stival procedure can detect rigidity and provide a good estimate of the fundamental matrix when the initial set of point matches contains up to 90% of outliers. RANSAC typically breaks down around 50% of outliers.

Moisan-Stival Procedure vs RANSAC

Relies on rigidity constraint - change of point view to apply a 3D rotation & translation to scene

Check points whether correlated by rigid motion

Define rigidity as the least epipolar distance for all geometric from 7 sub-pairs

Define meaningfulness as the expected number of similar sets in random (ask the question whether this set of points is epipolar by chance or not)

Requires no assumption on camera motion

Moisan-Stival + RANSAC

Moisan-Stival

Moisan-Stival + RANSAC

remove 2 more bad matches

Future work

Key point matches with multi view points

Fully illumination invariant (possible)

Global context with ASIFT to introduce more good matches when the image has many repetitive features

References

http://www.insa.ac.in/insa_pdf/20005b8c_49.pdf

http://www.ams.jhu.edu/~castello/362/Handouts/hungarian.pdf

http://people.cs.ubc.ca/~mariusm/uploads/FLANN/flann_visapp09.pdf

http://www.cmap.polytechnique.fr/~yu/publications/SIAM_ASIFT.pdf

http://www.math-info.univ-paris5.fr/~moisan/papers/CMLA2002-11.pdf

thanks