Projective 3D geometry class 4

Multiple View GeometryComp 290-089Marc Pollefeys

Content

•

Background: Projective geometry

(2D, 3D), Parameter estimation, Algorithm evaluation.

•

Single View: Camera model, Calibration, Single View Geometry.

•

Two Views: Epipolar

Geometry, 3D reconstruction, Computing F, Computing structure, Plane and homographies.

•

Three Views: Trifocal Tensor, Computing T.•

More Views: N-Linearities, Multiple view reconstruction, Bundle adjustment, auto-

calibration, Dynamic SfM, Cheirality, Duality

Multiple View Geometry course schedule

(subject to change)

Jan. 7, 9 Intro & motivation Projective 2D Geometry

Jan. 14, 16 (no course) Projective 2D Geometry

Jan. 21, 23 Projective 3D Geometry Parameter Estimation

Jan. 28, 30 Parameter Estimation Algorithm Evaluation

Feb. 4, 6 Camera Models Camera Calibration

Feb. 11, 13 Single View Geometry Epipolar Geometry

Feb. 18, 20 3D reconstruction Fund. Matrix Comp.

Feb. 25, 27 Structure Comp. Planes & Homographies

Mar. 4, 6 Trifocal Tensor Three View Reconstruction

Mar. 18, 20 Multiple View Geometry MultipleView Reconstruction

Mar. 25, 27 Bundle adjustment Papers

Apr. 1, 3 Auto-Calibration Papers

Apr. 8, 10 Dynamic SfM Papers

Apr. 15, 17 Cheirality Papers

Apr. 22, 24 Duality Project Demos

Last week …

( )T1,0,0l =∞

TT JIIJ* +=∞C⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=∞

000010001

*C

0ml * =∞CT (orthogonality)

circular points (similarities)

line at infinity (affinities)

Last week …pole-polar relation

0xy =CT

0222 =±± wyx εδ

conjugate points & lines

projective conic classification

affine conic classification

xl C= lx *C=

0lm * =CT

A B

C

DX

Chasles’

theorem

cross-ratio

Fixed points and lines

λee =H (eigenvectors H =fixed points)

lλl =−TH (eigenvectors H-T

=fixed lines)

(λ1

=λ2

⇒ pointwise fixed line)

Singular Value DecompositionT

nnnmmmnm ×××× = VΣUA

IUU =T

021 ≥≥≥≥ nσσσ L

IVV =T

nm ≥

XXVT XVΣ T XVUΣ T

TTTnnn VUVUVUA 222111 σσσ +++= L

⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

=

000

00

0000

Σ

2

1

L

MMM

L

MOMM

L

L

nσ

σσ

Singular Value Decomposition

•

Homogeneous least-squares

•

Span and null-space

•

Closest rank r approximation

•

Pseudo inverse

1X AXmin subject to = nVX solution =

( )nrrdiag σσσσσ ,,,,,, 121 LL +=Σ ( ) 0 ,, 0 ,,,,~21 LL rdiag σσσ=ΣTVΣ~UA~ = TVUΣA =

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

00000000000000

Σ 2

1

σσ

[ ] [ ]4321 UU;UU == LL NS[ ] [ ]4321 VV;VV == RR NS

TUVΣA ++ = ( ) 0 ,, 0 ,,,, 112

11 LL −−−+ =Σ rdiag σσσ

TVUΣA =

Projective 3D Geometry

•

Points, lines, planes and quadrics

•

Transformations

•

П∞

, ω∞

and Ω

∞

3D points

( )TT

1 ,,,1,,,X4

3

4

2

4

1 ZYXXX

XX

XX

=⎟⎟⎠

⎞⎜⎜⎝

⎛=

in R3

( )04 ≠X

( )TZYX ,,

in P3

XX' H= (4x4-1=15 dof)

projective transformation

3D point

( )T4321 ,,,X XXXX=

Planes

0ππππ 4321 =+++ ZYX

0ππππ 44332211 =+++ XXXX

0Xπ =T

Dual: points ↔ planes, lines ↔ lines

3D plane

0X~.n =+ d ( )T321 π,π,πn = ( )TZYX ,,X~ =14 =Xd=4π

Euclidean representation

n/d

XX' H=ππ' -TH=

Transformation

Planes from points

0πXXX

3

2

1

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

T

T

T

2341DX( )T123124134234 ,,,π DDDD −−=

( ) ( ) ( )( ) ( ) ( )( ) ( ) ( )( ) ( ) ( )

0det

4342414

3332313

2322212

1312111

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

XXXXXXXXXXXXXXXX

0πX 0πX 0,πX π 321 === TTT andfromSolve

(solve as right nullspace of )π⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

T

T

T

3

2

1

XXX

[ ] 0XXX Xdet 321 =

Or implicitly from coplanarity condition

124313422341 DXDXDX +− 01234124313422341 =−+− DXDXDXDX 13422341 DXDX −

Points from planes

0Xπππ

3

2

1

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

T

T

T

xX M= [ ]321 XXX=M0π =MT

⎥⎦

⎤⎢⎣

⎡=

Ip

M

( )Tdcba ,,,π =T

⎟⎠⎞

⎜⎝⎛= −−−

ad

ac

abp ,,

0Xπ 0Xπ 0,Xπ X 321 === TTT andfromSolve

(solve as right nullspace of )X⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

T

T

T

3

2

1

πππ

Representing a plane by its span

Lines

⎥⎦

⎤⎢⎣

⎡= T

T

BA

W μBλA+

⎥⎦

⎤⎢⎣

⎡= T

T

QP

W* μQλP+

22** 0WWWW ×==TT

⎥⎦

⎤⎢⎣

⎡=

00011000

W ⎥⎦

⎤⎢⎣

⎡=

00100100

W*

Example:

X-axis

(4dof)

Points, lines and planes

⎥⎦

⎤⎢⎣

⎡= TX

WM 0π =M

⎥⎦

⎤⎢⎣

⎡= Tπ

W*

M 0X =M

W

X

*Wπ

Plücker matrices

jijiij ABBAl −=TT BAABL −=

Plücker matrix (4x4 skew-symmetric homogeneous matrix)

1.

L

has rank 22.

4dof3.

generalization of4.

L

independent of choice A

and B5.

Transformation

24* 0LW ×=T

yxl ×=

THLHL'=

[ ] [ ]⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡ −

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

0001000000001000

1000

0001

0001

1000

L T

Example: x-axis

Plücker matrices

TT QPPQL* −=

Dual Plücker matrix L*

-1TLHHL -'* =

*12

*13

*14

*23

*42

*34344223141312 :::::::::: llllllllllll =

XLπ *=

LπX =

Correspondence

Join and incidence

0XL* =

(plane through point and line)

(point on line)

(intersection point of plane and line)

(line in plane)0Lπ =

[ ] 0π,L,L 21 =K (coplanar lines)

Plücker line coordinates[ ]T344223141312 ,,,,, llllll= 5P∈

[ ] 0

000001000010000100001000010000100000

34

42

23

14

13

12

344223141312 =

⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

llllll

llllll

( ) ( )B,A,BA,ˆ, ↔

[ ]( )ˆ|Kˆ

ˆˆˆˆˆˆB,AB,A,det 123413421423231442133412

==

+++++=T

llllllllllll

0231442133412 =++ llllll on Klein quadric 0K =T

Plücker line coordinates

( ) [ ] 0B,AB,A,detˆ| ==

( ) [ ] 0Q,PQ,P,detˆ| ==

( ) ( )( ) ( )( ) 0BPAQBQAPˆ| =−= TTTT

( ) 0| = (Plücker internal constraint)

(two lines intersect)

(two lines intersect)

(two lines intersect)

Quadrics and dual quadrics

(Q

: 4x4 symmetric matrix)0QXX =T

1.

9 d.o.f.2.

in general 9 points define quadric 3.

det Q=0 ↔ degenerate quadric4.

pole –

polar 5.

(plane ∩

quadric)=conic6.

transformation

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

••••••••••

=

ooo

oo

oQ

QXπ =QMMC T= MxX:π =

-1-TQHHQ'=

0πQπ * =T

-1* QQ =1.

relation to quadric (non-degenerate)2.

transformation THHQQ' ** =

Quadric classification

Rank Sign. Diagonal Equation Realization4 4 (1,1,1,1) X2+ Y2+ Z2+1=0 No real points

2 (1,1,1,-1) X2+ Y2+ Z2=1 Sphere0 (1,1,-1,-1) X2+ Y2= Z2+1 Hyperboloid (1S)

3 3 (1,1,1,0) X2+ Y2+ Z2=0 Single point1 (1,1,-1,0) X2+ Y2= Z2 Cone

2 2 (1,1,0,0) X2+ Y2= 0 Single line0 (1,-1,0,0) X2= Y2 Two planes

1 1 (1,0,0,0) X2=0 Single plane

Quadric classificationProjectively equivalent to sphere:

hyperboloid of two sheets

paraboloidsphere ellipsoid

Degenerate ruled quadrics:

cone two planes

Ruled quadrics:

hyperboloids of one sheet

Twisted cubic

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

++++++

=⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛=

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

2333231

2232221

2131211

23

2

1 1A

θθθθθθ

θθ

aaaaaaaaa

xxx

conic

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

++++++++++++

=

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

=

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

344

2434241

334

2333231

324

2232221

314

2131211

3

2

4

3

2

1 1

A

θθθθθθθθθθθθ

θθθ

aaaaaaaaaaaaaaaa

xxxx

twisted cubic

1.

3 intersection with plane (in general)2.

12 dof (15 for A –

3 for reparametrisation (1 θ θ2θ3)3.

2 constraints per point on cubic, defined by 6 points4.

projectively equivalent to (1 θ θ2θ3)5.

Horopter & degenerate case for reconstruction

Hierarchy of transformations

⎥⎦

⎤⎢⎣

⎡vTvtAProjective

15dof

Affine12dof

Similarity7dof

Euclidean6dof

Intersection and tangency

Parallellism of planes,Volume ratios, centroids,The plane at infinity π∞

The absolute conic Ω∞

Volume

⎥⎦

⎤⎢⎣

⎡10tA

T

⎥⎦

⎤⎢⎣

⎡10tR

T

s

⎥⎦

⎤⎢⎣

⎡10tR

T

Screw decompositionAny particular translation and rotation is

equivalent to a rotation about a screw axis and a translation along the screw axis.

2D Euclidean Motion and the screw decomposition

⊥+= ttt //

screw axis // rotation axis

The plane at infinity

∞

−

∞−

∞ =

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

⎥⎦

⎤⎢⎣

⎡

−==′ π

1000

1t0

ππA

AH

TT

A

The plane at infinity π∞

is a fixed plane under a projective transformation H iff H is an affinity

1.

canical position2.

contains directions 3.

two planes are parallel ⇔ line of intersection in π∞4.

line // line (or plane) ⇔ point of intersection in π∞

( )T1,0,0,0π =∞

( )T0,,,D 321 XXX=

The absolute conic

The absolute conic Ω∞

is a fixed conic under the projective transformation H iff H is a similarity

04

23

22

21 =

⎭⎬⎫++

XXXX

The absolute conic Ω∞

is a (point) conic on π∞

. In a metric frame:

( ) ( )T321321 ,,I,, XXXXXXor conic for directions:(with no real points)

1.

Ω∞

is only fixed as a set2.

Circle intersect Ω∞

in two points3.

Spheres intersect π∞

in Ω∞

The absolute conic( )

( )( )2211

21

ddddddcos

TT

T

=θ

( )( )( )2211

21

ddddddcos

∞∞

∞

ΩΩ

Ω=

TT

T

θ

0dd 21 =Ω∞T

Euclidean:

Projective:

(orthogonality=conjugacy)

plane

normal

The absolute dual quadric

⎥⎦

⎤⎢⎣

⎡=Ω∞ 00

0I*T

The absolute conic Ω*∞

is a fixed conic under the projective transformation H iff H is a similarity

1.

8 dof2.

plane at infinity π∞

is the nullvector of 3.

Angles:

( )( )2*

21*

1

2*

1

ππππππcos

∞∞

∞

ΩΩ

Ω=

TT

T

θ

*∞Ω

Next classes: Parameter estimation

Direct Linear TransformIterative EstimationMaximum Likelihood Est.Robust Estimation