16 Feature Alignment

7/30/2019 16 Feature Alignment

1/30

Automatic Image Alignment (feature-based)

15-463: Computational PhotographyAlexei Efros, CMU, Fall 2005with a lot of slides stolen fromSteve Seitz and Rick Szeliski

Mike Nese


2/30

Todays lecture

Feature detectors scale invariant Harris corners

Feature descriptors patches, oriented patches

Reading for Project #4:Multi-image Matching using Multi-scale image

patches, CVPR 2005


3/30

Invariant Local Features

Image content is transformed into local feature coordinates that areinvariant to translation, rotation, scale, and other imaging parameters

Features Descriptors


4/30

Advantages of local features

Locality: features are local, so robust to occlusion andclutter (no prior segmentation)

Distinctiveness: individual features can be matched to

a large database of objectsQuantity: many features can be generated for even

small objects

Efficiency: close to real-time performanceExtensibility: can easily be extended to wide range of

differing feature types, with each adding robustness


5/30

More motivation

Feature points are used for: Image alignment (homography, fundamental

matrix)

3D reconstruction Motion tracking Object recognition Indexing and database retrieval Robot navigation other


6/30

Harris corner detector

C.Harris, M.Stephens. A Combined Corner and EdgeDetector. 1988


7/30

The Basic Idea

We should easily recognize the point by lookingthrough a small window

Shifting a window in any direction should give a largechange in intensity


8/30

Harris Detector: Basic Idea

flat region:no change inall directions

edge :no change alongthe edge direction

corner :significant changein all directions


9/30

Harris Detector: Mathematics

[ ]2,

( , ) ( , ) ( , ) ( , ) x y

E u v w x y I x u y v I x y= + + Change of intensity for the shift [ u,v]:

IntensityShiftedintensity

Windowfunction

or Window function w(x,y) =

Gaussian1 in window, 0 outside


10/30


[ ]( , ) ,

u E u v u v M

v

For small shifts [u,v ] we have a bilinear approximation:

2

2,

( , ) x x y x y x y y

I I I M w x y

I I I

=

where M is a 2 2 matrix computed from image derivatives:


11/30


1

2Corner1 and 2 are large,1 ~ 2;

E increases in alldirections

1 and 2 are small; E is almost constantin all directions

Edge1 >> 2

Edge2 >> 1

Flatregion

Classification of image points usingeigenvalues of M :


12/30


Measure of corner response:

1 2

1 2

det

trace

M

M

=

= +

M M

RTrace

det=


13/30

Harris Detector

The Algorithm: Find points with large corner response function R

(R > threshold)

Take the points of local maxima of R


14/30

Harris Detector: Workflow


15/30


Compute corner response R


16/30


Find points with large corner response: R> threshold

kfl


17/30


Take only the points of local maxima of R

H i D W kfl


18/30



19/30

Harris Detector: Some Properties

Rotation invariance

Ellipse rotates but its shape (i.e. eigenvalues)remains the same

Corner response R is invariant to image rotation


20/30


Partial invariance to affine intensity change

9 Only derivatives are used => invarianceto intensity shift I I + b

9 Intensity scale: I a I

R

x (image coordinate)

threshold

R

x (image coordinate)


21/30


But: non-invariant to image scale !

All points will be

classified as edges

Corner !


22/30

Scale Invariant Detection

Consider regions (e.g. circles) of different sizes around a pointRegions of corresponding sizes will look the same in both images


23/30

Scale Invariant Detection

The problem: how do we choose corresponding circlesindependently in each image?

Choose the scale of the bestcorner

l


24/30

Feature selectionDistribute points evenly over the image

d i i lS i


25/30

Adaptive Non-maximal SuppressionDesired: Fixed # of features per image

Want evenly distributed spatially Search over non-maximal suppression radius

[Brown, Szeliski, Winder, CVPR05]

F d i


26/30

Feature descriptors

We know how to detect pointsNext question: How to match them?

?Point descriptor should be:

1. Invariant 2. Distinctive


27/30

Descriptors Invariant to Rotation

Find local orientation

Dominant direction of gradient

Extract image patches relative to this orientation

M lti S l O i t d P t h


28/30

Multi-Scale Oriented Patches

Interest points Multi-scale Harris corners Orientation from blurred gradient Geometrically invariant to rotation

Descriptor vector Bias/gain normalized sampling of local patch (8x8) Photometrically invariant to affine changes in intensity

[Brown, Szeliski, Winder, CVPR2005]

D i t V t


29/30

Descriptor Vector

Orientation = blurred gradientRotation Invariant Frame Scale-space position (x, y, s) + orientation ( )

Detections at multiple scales


30/30

Detections at multiple scales

Documents

16 Feature Alignment