Estimation- 2D Projective Transformations

1

Estimation-2D 1

2008-1

Multi View Geometry (Spring '08)

Prof. Kyoung Mu LeeSoEECS, Seoul National University

Estimation- 2D Projective Transformations

Estimation-2D 2

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

• Any real mxn matrix A can be decomposed uniquely by

where

SVD (Singular Value Decomposition)

mm×= IUUT

021 ≥≥≥≥ nddd L

nn×= IVVT

⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

=

000

00

0000

2

1

L

MMM

L

MOMM

L

L

nd

dd

D

Tnnnmmmnm ×××× = VDUA nm ≥

[ ][ ]n

mn

vvVuuuU

K

LK

,,

1

1

== orthonormal column vectors

2

Estimation-2D 3



• If we consider only the non-zero terms, we can write it by

• Properties:

•

•

∑=

=+++=

+++==n

iiiinnn

nnnT

dddd

ddd

1222111

222111

TTTT

TTT

vuvuvuvu

vuvuvuUDVA

L

L

2

2

⎢⎣

⎡

==

⇒

==

iiiT

iiiT

TTTT

dd

vAvAuuAA

UUDVDUUDVAA 2

Tnnnnnmnm ×××× = VDUA nm ≥

Estimation-2D 4



•

• For square, non-singular matrices,

• Closest rank r approximation

( ) 0 ,, 0 ,,,,~21 LL rddddiag=D

TTT

T

rrrddd vuvuvuVDUA

+++=

=

L222111

~~

=Ax

DVAx

xDVxVVDxAxAxAx

and by affectedonly is

222

⇒

=== TTTTT

TUVDA 11 −− = ( ) /1,,/1 11

ndddiag L=−D

xxVTxDVTxUDVT

3

Estimation-2D 5



• Solution of linear equations: • Least squares solution: Find x that minimize

i) if rank Amxn = n (full column rank)bAx−

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

′

′′

′′

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

⇒

=′=′−⇒

−=−=−

+

m

n

n

n

n

TT

TTT

b

bb

bb

y

yy

d

dd

y

M

M

M

O

1

2

1

2

1

2

1

0

, ,min

since

bUbxVbDy

bUxDVbxUDVbAx

Vyx =

=′

=⇒

And

,1 , nidby

i

ii L

min

bxA =×nm

(unique solution)

Estimation-2D 6



ii) if rank Amxn = r < n (not full column rank)

• Pseudo inverse

nnrr vvVyx λλ +++= ++ K11

TUVDA ++ = ( ) 0 ,, 0 ,,,, 112

11 LL −−−+ = rddddiagD

bAxbxA +× =→=nm

homogeneous solution

4

Estimation-2D 7



• Solution of homogeneous equations • Least squares solution:

Find x that minimize

i) if rank Amxn = n (full col. rank)

ii) if rank Amxn = r < n (x = null-vector of A)

0=× xA nm

1 subject to =xAx

,1 subject to min

and , since

xVyyDy

xVxxDVxUDVAxT

TTT

==⇒

===

nvVyxy

==⇒= T,0,1)(0,0,K

nnrr

Tnr

λλ),λ,λ,,(

vvVyxy

++==⇒=

++

+

K

KK

11

1,00

(the last e-vector)

Estimation-2D 8



• Another interpretation

Which x makes ? i) if

ii) if

xvuxvuxvuxUDVAx TTTnnn

T ddd +++== L222111

1 subject to minimum be to =xAx,n,idi K1 ,0 =≠

n

n

d is minimum thevector)-elast (the ,

⇒=⇒ vx

,n,ridi K1 ,0 +==

0 ) of space nullor vectors,-esingular theofspan (the

,11

=⇒

++==⇒ ++

AxA

vvVyx nnrr λλ K

5

Estimation-2D 9


Parameter Estimation

• We are interested in the following problems:

2D homographyGiven a set of (xi,xi’), how to determine H (xi’=Hxi)

3D to 2D camera projectionGiven a set of (Xi,xi), how to determine P (xi=PXi)

Fundamental matrixGiven a set of (xi,xi’), how to determine F (xi’TFxi=0)

Trifocal tensor Given a set of (xi,xi’,xi”), how to determine T

Estimation-2D 10


Number of measurements required• To estimate the parameters, we need

# of independent equations ≥ degrees of freedom• Example:

To estimate H:

H has 8 DOF2 independent equations / a point correspondence ( )4x2≥8So, at least 4 point correspondences are required

Hxx ='

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡′′

11λ

333231

232221

131211

yx

hhhhhhhhh

yx

ii xx ′↔

6

Estimation-2D 11


Approximate solutions

• Minimal solution4 points an exact solution for H

• More pointsNo exact solution, because measurements are inexact (“noise”)Find the optimal solution according to some cost functionAlgebraic or geometric/statistical cost

Estimation-2D 12


Gold Standard algorithm

• Cost function that is optimal for some assumptions• Computational algorithm that minimizes it is called “Gold

Standard” algorithm • Other algorithms can then be compared to it

7

Estimation-2D 13


The Direct Linear Transformation (DLT) algorithm

• Given a set of (noisy) point correspondences , how to compute H robustly for ?

• Note that H must satisfy

• Let and

• Then

ii xx ′↔

ii Hxx =′

0=×′ ii Hxx

),,( iiii yx ω′′′=′x

0=hAi

⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜

⎝

⎛

=

i

i

i

i

xhxhxh

HxT

T

T

3

2

1

0hhh

0xxx0x

xx0=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

′′−

′−′′′−

3

2

1

TTii

Tii

Tii

TTii

Tii

Tii

T

xyx

yω

ω

0xhxhxhxhxhxh

Hxx =⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

′−′′−′′−′

=×′

iT

iiT

i

iT

iiT

i

iT

iiT

i

ii

yxx

y

12

31

23

ωω

Estimation-2D 14


The DLT algorithm (cont.)

• Since 3rd row is redundant, we have

Two lin. independent Eqs. for each point correspondence.• Thus, 4 point correspondences are enough.• For more than 4 point correspondences

(Over-determined System)

0hA =i for noisy measurementii ehA =

0hhh

x0xxx0

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎦

⎤⎢⎣

⎡

−−

3

2

1

~~

Tii

TTi

Tii

Ti

T

xy )~,~ (where

i

ii

i

ii

yyxxωω ′′

=′′

=

inhomogeneous mapped point

0hhh

x0xxx0

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎦

⎤⎢⎣

⎡′−′′′−

3

2

1

Tii

TTii

Tii

Tii

T

xy

ωω

8

Estimation-2D 15



• 4 point case:Solve the exact solution H satisfying

The size of A is 8x9 or 12x9 (rank is 8)1-D null-space h is the nontrivial solutionsChoose the one with

0Ah =0h

AAAA

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

4

3

2

1

1=h

Estimation-2D 16



• More points case:Over-determined solution satisfying

No exact solution due to the “noise”So, find the approximate solution

Additional constraint needed for nontrivial solution e.g. 1=h

0Ah =0h

A

AA

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

n

2

1

M

Ahhh

min=∗

9

Estimation-2D 17


ObjectiveGiven n≥4 2D to 2D point correspondences {xi↔xi’}, determine the 2D homography matrix H such that xi’=Hxi

Algorithm(i) For each correspondence xi ↔xi’ compute Ai. Usually only

two first rows needed.

(ii) Assemble n 2x9 matrices Ai into a single 2nx9 matrix A(iii) Obtain SVD of A. Solution for h is last column of V(iv) Determine H from h


Estimation-2D 18


• By setting hj=1 (e.g. h9=1), and solve for 8-vector

• It can be solved by Gaussian elimination (4 points) or linear least-squares (more than 4 points)

• However, if h9=0 this approach fails, and also gives poor results if h9≈ 0

• Therefore, this approach is not recommended• Note h9=H33=0 if origin is mapped to infinity

Inhomogeneous solution

⎟⎟⎠

⎞⎜⎜⎝

⎛−=⎥

⎦

⎤⎢⎣

⎡ −−−''~

''000''''''''000

ii

ii

iiiiiiiiii

iiiiiiiiii

xwyw

xyxxwwwywxyyyxwwwywx

h

h~

( ) [ ] 0100

1000 =⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=∞ HHxlT

10

Estimation-2D 19


• 2D homographies (8dof)Minimum of 4 points or lines

• 3D homographies (15 dof)Minimum of 5 points or 5 planes

• 2D affinities (6 dof)Minimum of 3 points or lines

• How about mixed configurations?2 point and 2 lines (x)3 lines and 1 point (o)

Solutions from lines, etc.

ii lHl T=′ 0Ah =• in 2D

Point/line 2 constraintsConic 5 constraints

• in 3DPoint/plane 3 constraints

Estimation-2D 20


Cost functions

• Which cost function can we employ?

• Algebraic distance• Geometric distance

Transfer error, Symmetric transfer error, Reprojection error

• Sampson error

11

Estimation-2D 21


Algebraic distance error

• Residual (algebraic error) vector:Error associated with each correspondence

• The total algebraic distance error:

• Has no geometrical and statistical meaning• However, with good normalization it works satisfactory, so it

can be used for initialization

hAe ii =

ii xx ′↔

( )2

22alg x0x

xx0, heHxx ⎥

⎦

⎤⎢⎣

⎡′−′−

′−′−==′

TTT

TTT

iiii

iiiiiii xw

ywd

algebraic distance

( ) 2222alg , eAheHxx ===′ ∑∑

ii

iiid

( ) 22

21

221alg , aad +=xx where ( ) 21

T321 ,, xxa ×== aaa

Estimation-2D 22


Geometric distance error

• Transfer error (error in one (second) image)

• Symmetric transfer error (error in both images)

• Reprojection errorFind and pairs of perfectly matched points and simultaneously

x′ˆ

: measured value: estimated value: true value

xxx

H

( )2,argminˆii

i

d xHxHH

′= ∑

( ) ( )221- ,,argminˆiiii

i

dd HxxxHxHH

′+′= ∑

( ) ( ) ( )

ii

iiiii

ii ddii

xHx

xxxxxxHxxH

ˆˆˆ subject to

ˆ,ˆ,argminˆ,ˆ,ˆ 22

ˆ,ˆ,

=′

′′+=′ ∑′

x

e.g. calibration patterndistanceEuclidean : ),( yxd

12

Estimation-2D 23


Comparison of symmetric transfer error and reprojection error

( ) ( )221- ,, HxxxHx ′+′ dd

( ) ( )22 ˆ,ˆ, xxxx ′′+ dd

Estimation-2D 24


Comparison of geometric and algebraic distances

• For one image case,

• The algebraic error vector:

• The geometric distance:

• For 2D affine transform case;

Thus geometric distance is equal to algebraic distance

⎟⎟⎠

⎞⎜⎜⎝

⎛′′−′′′′−′′

=⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

⎥⎦

⎤⎢⎣

⎡′−′′′−

==iiii

iiii

iiii

iiiiii xx

yyx

yωω

ωωω

ωˆˆˆˆ

00

3

2

1

hhh

xxxx

ehA TTT

TTT

1ˆ then ,1 if =′=′= iii ωωω

⇒′′′==′′′′=′ Tiiii

Tiiii yxyx )ˆ,ˆ,ˆ(ˆ and ,),,(Let ωω xHxx

( ) ( ) ( )222iialg ˆˆˆˆˆ, iiiiiiii xxyyd ωωωω ′′−′′+′′−′′=′′ xx

( ) ( ) ( )( )( ) ii

iiiiiiii

dxxyyd

ωωωωωω

ˆ/ˆ,/ˆ/ˆˆ/ˆ/ˆ,

iialg

2/1222ii

′′′′=

′′−′′+′′−′′=′′

xxxx

13

Estimation-2D 25


Geometric interpretation of reprojection error

• Estimating H is to fit surface VH to the measurement points, in R4

• Let be the closest point, then

• Conic analogue:

Tiiiii yxyx ),,,( ′′=X

Tiiiii yxyx )ˆ,ˆ,ˆ,ˆ(ˆ ′′=X

( ) ( ) ( ) ( )( ) ( )22

22222

ˆ,ˆ,

ˆˆˆˆˆ

iiii

iiiiiiiiii

dd

yyxxyyxx

xxxx

XX

′′+=

′−′+′−′+−+−=−

( ) ( ) ( )2H22 ,ˆ,ˆ, νXxxxx iiiii ddd ⊥=′′+

Estimation-2D 26


Sampson error

• Close approximation to geometric error (between algebraic and geometric error)

• 1st order approximation of • For given H and any point on VH satisfies

• Linearize it by a Taylor series expansion

• Find lying on VH so that

•

XTyxyx ),,,( ′′=X

0X0Ah =⇔= )(HC

XδXX +=ˆ

εJδ

0δX

X0X

X

X

−=⇒

=∂∂

+⇒=

)()ˆ( HHH

CCC

X

( ) ( ) XH

HXH δX

XδX∂∂

+=+CCC

εJδδδ XXX −= subject to minimizes that vector theFind

14

Estimation-2D 27


Sampson error (cont.)

• Use of Lagrange method:minimize

Then The Sampson error: Overall error:

( )εJδδδ += XXX 2λ-TE

εJJJδεJJλελJJ

0εJδ0λ

λJδ0Jλδ0δ

X

X

XXX

1

1

)( And)( Thus,

22

−

−

−=

−=⇒−=

=+⇒=∂∂

=⇒=−⇒=∂∂

TT

TT

TTTTT

E

E

XδXX +=ˆ

( ) εJJεδδδ 1TTX

TX

2X −

==

Estimation-2D 28


Maximum Likelihood estimation (MLE)

• Optimal estimator in statistical sense• Assume iid mean-zero isotropic Gaussian measurement error

(outlier removed):

• Error in one image:

MLE:• Error in both images

MLE: minimize

( ) ( ) ( )22 2/,2πσ2

1Pr σxxx de−=

2),( Minimize)|}Pr({log Maximize iii

i d xHxHx ′⇒′ ∑

{ }( ) ( ) ( )22i 2, /

2i πσ2

1|Pr σidei

xHxHx ′−Π=′

{ }( ) ( ) ( )( ) ( )222i 2(, /

2iii πσ2

1}{,|,Pr σiii ddei

xHxx XxHxx ′+−Π=′

( ) ( ) iiii dd xHxxxxx ˆˆˆ with ,ˆ,ˆ, 2i

2i =′′′+∑

15

Estimation-2D 29


Mahalanobis distance

• Consider non-isotropic Gaussian model• Density normalized distance.

Measurement X with covariance matrix Σ

Error in two images (independent)

Varying covariances

( ) ( )XXΣXXXX −−=− −

Σ

1T2

22

Σ′Σ′−′+− XXXX

22

iiii

iii Σ′Σ

′−′+−∑ XXXX

Estimation-2D 30


Non-invariance of DLT

• Let

eg. Coordinate transformation

• Does the DLT algorithm applied to the correspondence setyield the transformation ?

1

~

~ and

~ ,~ where,~~ and ,

−′=

′′=′=′′ ↔↔HTTH

xTxTxxxxxxHH

iiiiiiii

ii xx ~~ ′↔ 1~ −′= HTTH

)~~~~( 1iiii xHxHTTHxTxTx =′=′=′′=′ −

16

Estimation-2D 31



• Effect of change of coordinates on algebraic error

• For similarity transform

( ) iiiii TxHTTxTxHxe -1~~~~ ′×′′=×′= ( ) iii eTHxxT ** ′=×′′=

( ) ( ) ( ) hAhA iiiiiiii seeseese,e ==== TTT212121 ,,R~~~~

⎥⎦

⎤⎢⎣

⎡=′

1sR0

tT ⎥

⎦

⎤⎢⎣

⎡=′

ss

Rt0R

T T*

-

( ) ( )iiii sdd HxxxHx ,~~,~algalg ′=′

cofactor matrix of T’

Estimation-2D 32



• However

( )

( )( ) 1~ subject to~~,~minimize

1 subject to~~,~minimize

1 subject to,minimize

2alg

2alg

2alg

=′⇔

=′⇔

=′

∑

∑

∑

HxHx

HxHx

HHxx

iii

iii

iii

d

d

d

17

Estimation-2D 33


Invariance of geometric error

• Given

• If T and T’ are Euclidean transforms

• If T and T’ are Similarity transforms, then

• Thus, geometric error is invariant to Euclidean and similarity transformations

1

~

~ and

~ ,~ where,~~ and ,

−′=

′′=′=′′ ↔↔HTTH

xTxTxxxxxxHH

iiiiiiii

( ) ( ) ( )( )ii

iiiiii

sdddd

HxxHxTxTTxHTTxTxHx

,,,~~,~ -1

′=

′′′=′′′=′

( ) ( ) ( )( )ii

iiiiii

dddd

HxxHxTxTTxHTTxTxHx

,,,~~,~ -1

′=

′′′=′′′=′

Estimation-2D 34


Normalizing transformations

• How to make DLT invariant? Choice of coordinates?• Data normalization: (Essential step in the DLT algorithm)

Translate so that the centroid to be the originScale so that the average distance to the origin to bePerform independently on both images

– Improves accuracy– Invariant to scale and coordinate system

• Or apply

2

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡++++

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡+

+=

−

100)(2/)/(10)(2/0)/(1

1002/02/0

T

1

norm hwhhwhwwhw

hhwwhw

w,h: width and height of the image

18

Estimation-2D 35


Importance of normalization

0hhh

00011000

3

2

1

=⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

⎥⎦

⎤⎢⎣

⎡′−′−′−

′′′−′−′−

iiiiiii

iiiiiii

xyxxxyxyyyxyyx

~102 ~102 ~102 ~102 ~104 ~104 ~10211

Big difference in orders of magnitude unstable

Estimation-2D 36


Normalized DLT algorithm

ObjectiveGiven n≥4 2D to 2D point correspondences {xi↔xi’}, determine the 2D homography matrix H such that xi’=Hxi

Algorithm(i) Normalize points (ii) Apply DLT algorithm to and find (iii) Denormalize solution

,~~ii xx ′↔

inormiinormi~ ,~ xTxxTx ′′=′=

norm-1

norm~THTH ′=

H~

• What about the rotation?• Don’t we need normalize the rotation effect? Why?

HHRRH =′= −1~

19

Estimation-2D 37


Iterative minimization methods

• For minimization of geometric errorSlower than direct methods (DLT)Initialization and stopping criterion are neededNo guaranteed convergence, local minima

• Steps:Defining cost functionParameterizationFunction specificationInitializationIteration

Estimation-2D 38


Parameterization

• Parameters should cover complete space and allow the estimation to be efficient

• Minimal or over-parameterization? e.g. 8 or 9 minimal parameterization gives often more complicate functionsgood algorithms can deal with over-parameterizationSo, it is not necessary to use minimal parameterization

• Parameterization can also be used to restrict transformation to particular class, e.g. affine

20

Estimation-2D 39


Function Specification

• Function specification:

(i) Measurement vector X∈RN with covariance Σ(ii) Set of parameters represented by vector P ∈RM

(iii) Mapping f : RM →RN. Range of mapping is surface S representing allowable measurements

(iv) Cost function: squared Mahalanobis distance

• The goal is to achieve P such that , or get as close as possible in terms of Mahalanobis distance

( ) ( )( ) ( )( )PXΣPXPX fff −−=− −Σ

1T2

( ) XP =f

Estimation-2D 40


Iterative minimization methods (cont.)

• Error in one image:Let Then

• Symmetric transfer error:Let

Then

• Reprojection error:Let

Then

Tnn yxyxyx ),,,,,,( 2211 ′′′′′′= KX

22 ),()( ii

idf xHxhX ∑ ′⇔−

Tnnnn yxyxyxyx ),,,,,,,,,( 1111 ′′′′= KKX

2212 ),(),()( ii

iii ddf HxxxHxhX ∑ ′+′⇔− −

Tnnnn yxyxyxyx ),,,,,,,,,( 1111 ′′′′= KKX

222 )ˆ,()ˆ,()( ii

iii ddf xxxxPX ′′+⇔− ∑

( )nf Hx,...,Hx,Hxh: 21→

( )nnf HxHxHxxHxHxHh ,...,,,,...,,: 21-1

2-1

1-1 ′′′→

( ) ( )nnnf x,x,...,x,x,x,xx,...,x,xh,: 221121 ′′′→

21

Estimation-2D 41


Iterative minimization methods (cont.)

• Sampson approximation:Since it determines the 2n variables for each particular choice h, the minimization only requires the 9 parameters of h.

• Initialization:Good initialization is very important.Use linear solution (Normalized DLT) or random sampling (robust algorithm), etc.

• Iteration:Newton methodLM (Levenberg-Marquardt) methodPowell’s MethodSimplex method, etc.

}ˆ{ ix

Estimation-2D 42


The Gold Standard Algorithm

ObjectiveGiven n≥4 2D to 2D point correspondences {xi↔xi’}, determine the Maximum Likelyhood Estimation of H(this also implies computing optimal xi’=Hxi)

Algorithm(i) Initialization: compute an initial estimate using normalized

DLT or RANSAC (ii) Geometric minimization of -Either Sampson error:

• Minimize the Sampson error• Minimize using Levenberg-Marquardt over 9 entries of h

or Gold Standard error:• compute initial estimate for optimal {xi}• minimize cost over {H,x1,x2,…,xn}• if many points, use sparse method

( ) ( )22 ˆ,ˆ, iiii dd xxxx ′′+∑

22

Estimation-2D 43


Experimental Comparison

•

Estimation-2D 44


Robust Estimation-RANSAC

• How to determine inliers (discard outliers or mismatched correspondences) to estimate the homography robustly?

• RANSAC (RAndom SAmple Consensus):• Consider a 2D line-fitting example

Select two points randomly – determine a lineCheck the support points that lie within a distance thresholdThe line with most support is the robust fit.

baxx +=′ Estimation of 1-D affine transform

23

Estimation-2D 45


RANSAC

Estimation-2D 46


RANSAC (cont.)

ObjectiveRobust fit of model to data set S which contains outliers

Algorithm(i) Randomly select a sample of s data points from S and

instantiate the model from this subset.(ii) Determine the set of data points Si which are within a distance

threshold t of the model. The set Si is the consensus set of samples and defines the inliers of S.

(iii) If the subset of Si is greater than some threshold T, re-estimate the model using all the points in Si and terminate

(iv) If the size of Si is less than T, select a new subset and repeat the above.

(v) After N trials the largest consensus set Si is selected, and the model is re-estimated using all the points in the subset Si

24

Estimation-2D 47


RANSAC (cont.)

• What is the distance threshold?• The square of the point distance is a sum of squared

Gaussian variables, i.e., Chi-distribution with m degree of freedom (co-dimension m).

•• Thus, for a given probability α,

• And, for Gaussian with N(0,σ) we can decide

2mχ

2⊥d

∫==<2

0

2222 )()()Pr(k

mmm dkFk ζζχχ

)(

)()Pr()Pr(12

22222

α

αχα−

⊥

=⇒

==<⇒=<

m

mm

Ft

tFttd

(cdf)

chi distribution

chi-square distribution

21222

22

)( with outlier inlier

σα−

⊥

⊥ =⎩⎨⎧

<<

mFttdtd

Estimation-2D 48


RANSAC (cont.)

• How many samples?• Let p be the probability that at least one of the random

samples of s points among N selections is free from outliers, and ω be the probability that any selected point is an inlier.

• Then,

εωω −=−−=

−−=

−=

1 ,)1(1inliers)) are points s (allPr 1(1

outliers)includeselectionsN(allPr 1

Ns

N

p

7.81σ2T3

5.99σ2H,P2

3.84σ2l,F1

t 2ModelCodimension

95.0for )(12 == − ααmFt

25

Estimation-2D 49


RANSAC (cont.)

• Ex) line fitting case; n=12, ε=2/12=1/6, s=2. Thus N=5.

( ) ( )( )spN ε−−−= 11log/1log

11772727844269585881635433208472939737241674614657261712645723417139534351911974331711765322

50%40%30%25%20%10%5%sproportion of outliers e

Example of N for p=0.99 for a given s and ε

Estimation-2D 50


RANSAC (cont.)

• How large is an acceptable consensus set?• Terminate when the inlier ratio reaches to the expected

ratio of inliersFor n data points, T=(1- ε)nFor the line fitting example, T=(1- 0.2)12=10.

• Adaptive determination of the # of samples

26

Estimation-2D 51


Robust MLE

• Final step for RANSAC• Re-estimate the model using all the inliers.

A least-squares fit by minimizing

• Iterative estimation:Optimal fit to inliersRe-classify inliersIterate until no change

Robust cost function

∑ ⊥=i

id 2C

( ) ( )⎩⎨⎧

><

== ⊥∑ outlier inlier

with 222

222

i tettee

ed i γγD

Estimation-2D 52


Other Robust Estimation Algorithms

• RANSACMaximizes the # of inliers

• LMedS (Least Median Squares) estimationMinimizes the median errorScore the model by the median of the distances to all dataNo threshold or prior knowledge of error varianceHowever, fails if more than half of the data is outlyingUse the proportion if outliers to determine the distance

• Case deletion, Iterative least-squares Not recommended !

27

Estimation-2D 53


Automatic computation of a H using RANSAC

ObjectiveCompute homography between two images

Algorithm(i) Interest points: Compute interest points in each image(ii) Putative correspondences: Compute a set of interest point

matches based on some similarity measure(iii) RANSAC robust estimation: Repeat for N samples

(a) Select 4 correspondences and compute H(b) Calculate the distance d⊥ for each putative match(c) Compute the number of inliers consistent with H (d⊥<t)Choose H with most inliers

(iv) Optimal estimation: re-estimate H from all inliers by minimizing ML cost function with Levenberg-Marquardt

(v) Guided matching: Determine more matches using prediction by computed H

Optionally iterate last two steps until convergence

Estimation-2D 54


• Determining putative correspondences:Select corner points in each imagesMatch points using similarity measure (SAD, SSD, NCC, etc) symmetrically within a search region

• Distance measure:Symmetric transfer errorReprojection errorSampson error

• Sample selection:Avoid degenerate samples (collinear ones)Samples with good spatial distribution

• Robust ML estimation (cycle approach)Carry out ML estimation on inliers using Levenberg-Marquardt algorithmRecompute the inliers using new HIterate cycle until converges

Automatic computation of a H using RANSAC

28

Estimation-2D 55


Example: robust computation

Interest points(500/image)

Putative correspondences (268)

Outliers (117)

Inliers (151)

Final inliers (262)

Estimation-2D 56


Implementation and run details

• Harris corner detector is used• SSD threshold is set 20 (40 for the guided matching stage)• For 640 x 480 image, ROI of 80 x 80 is sufficient. • t=1.25• N=43, two MLE cycles, 10 iterations for LM algorithm

Documents

Estimation- 2D Projective Transformations