Class 51 Multi-linear Systems and Invariant Theory in the Context of Computer Vision and Graphics Class 5: Self Calibration CS329 Stanford University Amnon

Class 5 1

Multi-linear Systems and Invariant Theory

in the Context of Computer Vision and Graphics

Class 5: Self Calibration

CS329Stanford University

Amnon Shashua

Class 5 2

Material We Will Cover Today

• The basic equations and counting arguments

• The “absolute conic” and its image.

• Kruppa’s equations

• Recovering internal parameters.

Class 5 3

The Basic Equations and Counting Arguments

1

];[W

V

U

TRKp

Recall,

3D->2D from Euclidean world frame to image

Let K,K’ be the internal parameters of camera 1,2 and choose canonical

frame in which R=I and T=0 for first camera.

PKp ]0;[

PtRKp ][''

1

Z

Y

X

P

T

W

V

U

R

Z

Y

X

world frame to first camera frame

Class 5 4


p

WWIp 1]0;[

p

WZ

Y

X

1

1

where

1W maps from the projective frame to Euclidean

1

0Tn

KW

Z

Y

X

KZ

p1

(8 unknown parameters)

Class 5 5

)0,,,( ZYX are the points on the plane at infinity (in Euc frame)

)1,0,0,0( is the plane at infinity

1

0

0

0

TW is the plane at infinity in Proj frame

(recall: if W maps points to points (Euc -> Proj), then the dualTW maps planes to planes)


Class 5 6


11

1

0

0

0

10

1

0

0

0

unKnKKW

T

T

TTT

uKn T

Class 5 7


p

eHp '' Projective frame

p

WWeH 1'

p

Wn

KeH T

1

1

0'

p

WtRK 1'

1

0'' Tn

KeHtRK

Class 5 8

1

0'' Tn

KeHtRK


RKneHK T ''

RneKHKK T ''' 11

uKn Tsince then,

RKueKHKK T ''' 11

but provides 5 (non-linear) constraints!IRRT

Class 5 9

RKueKHKK T ''' 11

IRRT

TTTTT KKueHKKueH '')'()'(

Since the right-hand side is symmetric and up to scale, we have5 constraints.


Class 5 10


Lets do some counting:

Let 5#1 K be the number of internal parameters

m be the number of views

3))(#1()(#)1(5 KmKm

Class 5 11


3))(#1()(#)1(5 KmKm

5# K not enough measurements (!)

4# K )1(47)1(5 mm 8m

3# K )1(36)1(5 mm 4m

1# K )1(4)1(5 mm 2m

'....'' KKK 8)1(5 m 3m(fixed internal params)

Class 5 12


The remainder of this lecture is about a geometric insight of

Class 5 13

The Absolute Conic

0CppT where TCC represents a conic in 2D

)0,,,( 321 xxx are the points on the plane at infinity (in Euc frame)

)1,0,0,0( is the plane at infinity

0

3

2

1

3

2

1

x

x

x

C

x

x

xT

is conic on the plane at infinity

IC when 023

22

21 xxx

is the “absolute” conic (imaginary circle)

Class 5 14

Plane at infinity is preserved under affine transformations:

10TvA

W because

0

'

'

'

03

2

1

3

2

1

x

x

x

x

x

x

W

is preserved under similarity transformation (R,t up to scale)

3

2

1

3

2

1

0x

x

x

Ax

x

x

W if 0CppT and 'pAp

0)'(''' pACAppCp TTTthen

1' CAAC T

but ,IC so in order that IC ' we must have:

1 AAI T A is orthogonal

The Absolute Conic

Class 5 15

The Image of the Absolute Conic

3

2

1

3

2

1

0

][

x

x

x

KRx

x

x

tRKImage of points at infinity:

let KRA

if 0CppT is a conic on the plane at infinity

then 1' CAAC T is the projected conic onto the image

since ,IC then11)()(' KKKRKRC TT

the image of is

1 KK T

Class 5 16

The Image of the Dual Absolute Conic

0CppT Cpl is tangent to the conic at p

lCp 1 011 lCllCCClCpp TTTT

TKK* is the image of the dual absolute conic


The basic equation:

Becomes:

** ')'()'( TTT ueHueH

Why 8 parameters? 5 for the conic, 3 for the plane

Class 5 17

Geometric Interpretation of

p

d

Kdd

Kp

10

pKd 1 direction of

optical ray

10

0

0

1

d

The angle between two optical rays 21,dd

2211

21

21

21

||||)cos(

pppp

pp

dd

ddTT

TT

1 KK Tgiven one can measure angles

Class 5 18

Kruppa’s Equations

General idea: eliminate n from the basic equation.

C

'C

1l

'1l

2l'2l

e

'e 1p2p

'1p

'2p

0iTi Cpp

01 i

Ti lCl

ii pel ][

0][][ 1

itTii

Ti pCppeCep

0']'[']'[' '''1

itTii

Ti pCppeCep

', tt CC are degenerate (rank 2) conics

Class 5 19

Kruppa’s Equations Note:

TTt llllC 1221 is a degenerate conic

0pCp tT iff 01 lpT or 02 lpT

Let H be the homography induced by the plane of the conic

HCHC tT

t' (slide 14)

HeCeHeCe T

]'[']'[][][ 11

tC'tC

Class 5 20

HeCeHeCe T

]'[']'[][][ 11

Kruppa’s Equations

Recall: HeF ]'[

FCFeCe T 11 '][][

In our case and the conic is

FFee T ** '][][

FKKFeKKe TTT ''][][

Likewise: TFFee ** ]'[']'[

Class 5 21

Determining K given Recall:

1

u the location of the plane at inifinity in

the projective coordinate frame.

We wish to represent the homography H induced by

Let 0TP be a point on the plane at infinity.

p

P

0upT

up

pP T

pueHeupHpp

eHp TT )'(')(''

TueHH '

Class 5 22

** ')'()'( TTT ueHueH

Determining K given

Recall: (slide 16)

TKK* TKK '''*

** ' THH

Note:this could be derived from “first principles” as well:

0''' * ll T tangents lines to the image of the absolute conic

'llH T 0'*1

lHHl TT

*

0* llT

Class 5 23

Determining K given

** ' THH

Assume fixed internal parameters ** '

** THH

TKK*Provides 4 independent linear constraints on

Why 4 and not 5?

we need 3 views (since* has 5 unknowns)

1)det( H

Note: 1 KRKH1)det( H

TTTTT KKKRKKKKRK 1

Class 5 24

Why 4 Constraints?

** THH

1 KRKH1)det( H

H and Rare “similar” matrices, i.e., have the same eigenvalues

Let w be the axis of rotation, i.e., wRw

H has an eigenvalue = 1, with eigenvector

vvKRK 1( ))( 11 vKvKR

Kwv

Class 5 25

Why 4 Constraints?

vvH

** THHif

then

TTTT vHvHHHHvvH ))(()( **

* is a solution to

Tvv *is also a solution

Tvv *

We need one more camera motion (with a different axisof rotation).

Class 5 26

Kruppa’s Equations (revisited)

Kruppa’s equations:

** ' THH

Start with the basic equation:

Multiply the terms by

TFFee ** ]'[']'[

]'[e on both sides:

]'[]'[]'[']'[ ** eHHeee T

F

Documents

Class 51 Multi-linear Systems and Invariant Theory in the Context of Computer Vision and Graphics Class 5: Self Calibration CS329 Stanford University Amnon