Projective 3D geometryclass 4

Multiple View GeometryComp 290-089Marc Pollefeys

Content

• Background: Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation.

• Single View: Camera model, Calibration, Single View Geometry.

• Two Views: Epipolar Geometry, 3D reconstruction, Computing F, Computing structure, Plane and homographies.

• Three Views: Trifocal Tensor, Computing T.• More Views: N-Linearities, Multiple view

reconstruction, Bundle adjustment, auto-calibration, Dynamic SfM, Cheirality, Duality

Multiple View Geometry course schedule(subject to change)

Jan. 7, 9 Intro & motivation Projective 2D Geometry

Jan. 14, 16

(no course) Projective 2D Geometry

Jan. 21, 23

Projective 3D Geometry Parameter Estimation

Jan. 28, 30

Parameter Estimation Algorithm Evaluation

Feb. 4, 6 Camera Models Camera Calibration

Feb. 11, 13

Single View Geometry Epipolar Geometry

Feb. 18, 20

3D reconstruction Fund. Matrix Comp.

Feb. 25, 27

Structure Comp. Planes & Homographies

Mar. 4, 6 Trifocal Tensor Three View Reconstruction

Mar. 18, 20

Multiple View Geometry MultipleView Reconstruction

Mar. 25, 27

Bundle adjustment Papers

Apr. 1, 3 Auto-Calibration Papers

Apr. 8, 10

Dynamic SfM Papers

Apr. 15, 17

Cheirality Papers

Apr. 22, 24

Duality Project Demos

Last week …

T1,0,0l

TT JIIJ* C

000

010

001*C

0ml * CT

(orthogonality)

circular points (similarities)

line at infinity (affinities)

Last week …pole-polar relation

0xy CT

0222 wyx

conjugate points & lines

projective conic classification

affine conic classification

xl C lx *C

0lm * CT

A B

C

DX

Chasles’ theorem

cross-ratio

Fixed points and lines

λee H (eigenvectors H =fixed points)

lλl TH (eigenvectors H-T =fixed lines)

(1=2 pointwise fixed line)

Singular Value Decomposition

Tnnnmmmnm VΣUA

IUU T

021 n

IVV T

nm

XXVT XVΣ T XVUΣ T

TTTnnn VUVUVUA 222111

000

00

00

00

Σ

2

1

n

Singular Value Decomposition

• Homogeneous least-squares

• Span and null-space

• Closest rank r approximation

• Pseudo inverse

1X AXmin subject to nVX solution

nrrdiag ,,,,,, 121 0 ,, 0 ,,,,~

21 rdiag TVΣ~

UA~

TVUΣA

0000

0000

000

000

Σ 2

1

4321 UU;UU LL NS 4321 VV;VV RR NS

TUVΣA 0 ,, 0 ,,,, 112

11 rdiag

TVUΣA

Projective 3D Geometry

• Points, lines, planes and quadrics

• Transformations

• П∞, ω∞ and Ω ∞

3D points

TT

1 ,,,1,,,X4

3

4

2

4

1 ZYXX

X

X

X

X

X

in R3

04 X

TZYX ,,

in P3

XX' H (4x4-1=15 dof)

projective transformation

3D point

T4321 ,,,X XXXX

Planes

0ππππ 4321 ZYX

0ππππ 44332211 XXXX

0Xπ T

Dual: points ↔ planes, lines ↔ lines

3D plane

0X~

.n d T321 π,π,πn TZYX ,,X~

14 Xd4π

Euclidean representation

n/d

XX' Hππ' -TH

Transformation

Planes from points

0π

X

X

X

3

2

1

T

T

T

2341DX T123124134234 ,,,π DDDD

0det

4342414

3332313

2322212

1312111

XXXX

XXXX

XXXX

XXXX

0πX 0πX 0,πX π 321 TTT andfromSolve

(solve as right nullspace of )π

T

T

T

3

2

1

X

X

X

0XXX Xdet 321

Or implicitly from coplanarity condition

124313422341 DXDXDX 01234124313422341 DXDXDXDX 13422341 DXDX

Points from planes

0X

π

π

π

3

2

1

T

T

T

xX M 321 XXXM

0π MT

I

pM

Tdcba ,,,π T

a

d

a

c

a

bp ,,

0Xπ 0Xπ 0,Xπ X 321 TTT andfromSolve

(solve as right nullspace of )X

T

T

T

3

2

1

π

π

π

Representing a plane by its span

Lines

T

T

B

AW μBλA

T

T

Q

PW* μQλP

22** 0WWWW TT

0001

1000W

0010

0100W*

Example: X-axis

(4dof)

Points, lines and planes

TX

WM 0π M

Tπ

W*

M 0X M

W

X

*W

π

Plücker matrices

jijiij ABBAl TT BAABL

Plücker matrix (4x4 skew-symmetric homogeneous matrix)

1. L has rank 22. 4dof3. generalization of

4. L independent of choice A and B5. Transformation

24* 0LW T

yxl

THLHL'

0001

0000

0000

1000

1000

0

0

0

1

0001

1

0

0

0

L T

Example: x-axis

Plücker matrices

TT QPPQL*

Dual Plücker matrix L*

-1TLHHL -'*

*12

*13

*14

*23

*42

*34344223141312 :::::::::: llllllllllll

XLπ *

LπX

Correspondence

Join and incidence

0XL*

(plane through point and line)

(point on line)

(intersection point of plane and line)

(line in plane)0Lπ

0π,L,L 21 (coplanar lines)

Plücker line coordinates

T344223141312 ,,,,, llllll 5P

0

000001

000010

000100

001000

010000

100000

34

42

23

14

13

12

344223141312

l

l

l

l

l

l

llllll

B,A,BA,ˆ,

ˆ|Kˆ

ˆˆˆˆˆˆB,AB,A,det 123413421423231442133412

T

llllllllllll

0231442133412 llllll on Klein quadric 0K T

Plücker line coordinates

0B,AB,A,detˆ|

0Q,PQ,P,detˆ|

0BPAQBQAPˆ| TTTT

0| (Plücker internal constraint)

(two lines intersect)

(two lines intersect)

(two lines intersect)

Quadrics and dual quadrics

(Q : 4x4 symmetric matrix)0QXX T

1. 9 d.o.f.

2. in general 9 points define quadric

3. det Q=0 ↔ degenerate quadric

4. pole – polar

5. (plane ∩ quadric)=conic

6. transformation

Q

QXπ QMMC T MxX:π

-1-TQHHQ'

0πQπ * T

-1* QQ 1. relation to quadric (non-degenerate)

2. transformation THHQQ' **

Quadric classification

Rank Sign. Diagonal Equation Realization

4 4 (1,1,1,1) X2+ Y2+ Z2+1=0 No real points

2 (1,1,1,-1) X2+ Y2+ Z2=1 Sphere

0 (1,1,-1,-1) X2+ Y2= Z2+1 Hyperboloid (1S)

3 3 (1,1,1,0) X2+ Y2+ Z2=0 Single point

1 (1,1,-1,0) X2+ Y2= Z2 Cone

2 2 (1,1,0,0) X2+ Y2= 0 Single line

0 (1,-1,0,0) X2= Y2 Two planes

1 1 (1,0,0,0) X2=0 Single plane

Quadric classificationProjectively equivalent to sphere:

Ruled quadrics:

hyperboloids of one sheet

hyperboloid of two sheets

paraboloidsphere ellipsoid

Degenerate ruled quadrics:

cone two planes

Twisted cubic

2333231

2232221

2131211

23

2

1 1

A

aaa

aaa

aaa

x

x

x

conic

344

2434241

334

2333231

324

2232221

314

2131211

3

2

4

3

2

1 1

A

aaaa

aaaa

aaaa

aaaa

x

x

x

x

twisted cubic

1. 3 intersection with plane (in general)

2. 12 dof (15 for A – 3 for reparametrisation (1 θ θ2θ3)

3. 2 constraints per point on cubic, defined by 6 points

4. projectively equivalent to (1 θ θ2θ3)

5. Horopter & degenerate case for reconstruction

Hierarchy of transformations

vTv

tAProjective15dof

Affine12dof

Similarity7dof

Euclidean6dof

Intersection and tangency

Parallellism of planes,Volume ratios, centroids,The plane at infinity π∞

The absolute conic Ω∞

Volume

10

tAT

10

tRT

s

10

tRT

Screw decompositionAny particular translation and rotation is

equivalent to a rotation about a screw axis and a translation along the screw axis.

ttt //

screw axis // rotation axis

The plane at infinity

π

1

0

0

0

1t

0ππ

A

AH

TT

A

The plane at infinity π is a fixed plane under a projective transformation H iff H is an affinity

1. canical position2. contains directions 3. two planes are parallel line of intersection in π∞

4. line // line (or plane) point of intersection in π∞

T1,0,0,0π

T0,,,D 321 XXX

The absolute conic

The absolute conic Ω∞ is a fixed conic under the projective transformation H iff H is a similarity

04

23

22

21

X

XXX

The absolute conic Ω∞ is a (point) conic on π.

In a metric frame:

T321321 ,,I,, XXXXXXor conic for directions:(with no real points)

1. Ω∞ is only fixed as a set2. Circle intersect Ω∞ in two points3. Spheres intersect π∞ in Ω∞

The absolute conic

2211

21

dddd

ddcos

TT

T

2211

21

dddd

ddcos

TT

T

0dd 21 T

Euclidean:

Projective:

(orthogonality=conjugacy)

plane

normal

The absolute dual quadric

00

0I*T

The absolute conic Ω*∞ is a fixed conic under the

projective transformation H iff H is a similarity

1. 8 dof2. plane at infinity π∞ is the nullvector of Ω∞

3. Angles:

2*

21*

1

2*

1

ππππ

ππcos

TT

T

Next classes:Parameter estimation

Direct Linear TransformIterative EstimationMaximum Likelihood Est.Robust Estimation