View
230
Download
7
Embed Size (px)
Citation preview
Camera Models
CMPUT 498/613CMPUT 498/613
Richard Hartley and Andrew Zisserman, Multiple View Geometry, Cambridge University Publishers, 2000
Readings: HZ Ch 6, 7
Hierarchy of cameras
Camera center
Image plane Object plane
X0(origin)xpersp
Perspective:
xparap
Para-perspective:
First order approximation of perspective
z
y
x
persp
t
t
t
f
f
P
k
j
i
1
xorth
Orthographic:
1Ty
x
orth t
t
P
0
j
i
Weak perspective:
xwp
11 Ty
x
wp t
t
k
k
P
0
j
i
Examples of camera projections
perspective Orthographic (parallel)
Animal eye:
a looonnng time ago.
Pinhole perspective projection: Brunelleschi, XVth Century.Camera obscura: XVIth Century.
Photographic camera:Niepce, 1816.
TT ZfYZfXZYX )/,/(),,(
101
0
0
1
Z
Y
X
f
f
Z
fY
fX
Z
Y
X
Pinhole camera model
Pinhole camera model
101
0
0
Z
Y
X
f
f
Z
fY
fX
101
01
01
1Z
Y
X
f
f
Z
fY
fX
PXx
0|I)1,,(diagP ff
Principal point offset
Tyx
T pZfYpZfXZYX )/,/(),,(
principal pointT
yx pp ),(
101
0
0
1
Z
Y
X
pf
pf
Z
ZpfY
ZpfX
Z
Y
X
y
x
x
x
Principal point offset
101
0
0
Z
Y
X
pf
pf
Z
ZpfY
ZpfX
y
x
x
x
camX0|IKx
1y
x
pf
pf
K calibration matrix
Camera rotation and translation
C~
-X~
RX~
cam
X10
RCR
1
10
C~
RRXcam
Z
Y
X
camX0|IKx XC~
|IKRx
t|RKP C~
Rt PXx
CCD camera
1yx
xx
p
p
K
11y
x
x
x
pf
pf
m
m
K
Finite projective camera
1yy
xx
p
ps
K
C~
|IKRP
non-singular
11 dof (5+3+3)
decompose P in K,R,C?
4p|MP 41pMC
~ MRK, RQ{finite cameras}={P4x3 | det M≠0}
If rank P=3, but rank M<3, then cam at infinity
Camera anatomy
Camera centerColumn pointsPrincipal planeAxis planePrincipal pointPrincipal ray
Column vectors
0
0
1
0
ppppp 43212
Image points corresponding to X,Y,Z directions and origin
Row vectors
1p
p
p
0 3
2
1
Z
Y
X
y
x
T
T
T
1p
p
p0
3
2
1
Z
Y
X
w
yT
T
T
note: p1,p2 dependent on image reparametrization
Camera center
0PC null-space camera projection matrix, (3 plane intersection)
The principal point
principal point
0,,,p̂ 3332313 ppp
330 Mmp̂Px
Action of projective camera on point
PXx
MdDp|MPDx 4
Forward projection
Back-projection
xPX 1PPPP
TT IPP
(pseudo-inverse)
0PC
λCxPλX
1
p-μxM
1
pM-
0
xMμμX 4
-14
-1-1
xMd -1
CD
Camera matrix decomposition
Finding the camera center
0PC (use SVD to find null-space)
Finding the camera orientation and internal parameters
KRM (use RQ decomposition ~QR)
Q R=( )-1= -1 -1QR
(if only QR, invert)
When is skew non-zero?
1yy
xx
p
ps
K
1
arctan(1/s)
for CCD/CMOS, almost always s=0
Image from image, s≠0 possible(non coinciding principal axis)
HPresulting camera:
Euclidean vs. projective
homography 44
0100
0010
0001
homography 33P
general projective interpretation
Meaningfull decomposition in K,R,t requires Euclidean image and space
Camera center is still valid in projective space
Principal plane requires affine image and space
Principal ray requires affine image and Euclidean space
Cameras at infinity
00
dP
Camera center at infinity
0Mdet
Affine and non-affine cameras
Definition: affine camera has P3T=(0,0,0,1)
Affine cameras
Affine cameras
C~
rr
C~
rr
C~
rr
KC~
|IKRP3T3T
2T2T
1T1T
0
C~
r3T0 d
t
t
dt
t
t
3T
2T2T
1T1T
33T3T
32T2T
31T1T
r
C~
rr
C~
rr
K
r-C~
rr
r-C~
rr
r-C~
rr
KP
modifying p34 corresponds to moving along principal ray
Affine cameras
003T
2T2T
1T1T
0
3T
2T2T
1T1T
0
0
/r
C~
rr
C~
rr
K
r
C~
rr
C~
rr
1
/
/
KP
dddd
d
d
dd
dd
t
t
t
t
t
t
now adjust zoom to compensate
0
2T2T
1T1T
0
C~
rr
C~
rr
KPlimP
dt
t
Error in employing affine cameras
1
βrαrX
21
1
rβrαrX
321
XPXPXP t0
point on plane parallel with principal plane and through origin, then
general points
Δ
~
~
KXPx
0
0proj
d
y
x
0
affine~
~
KXPx
d
y
x
projxaffinex
0x
Affine imaging conditions
0proj0
projaffine x-xx-xd
Approximation should only cause small error
1. much smaller than d0
2. Points close to principal point (i.e. small field of view)
Decomposition of P∞
0
02x2
0
t~R~
10
x~KP
d
10
t~R~
10
x~K 02x2-1
0d
absorb d0 in K2x2
10
0R~
10
x~t~KK
10
x~Kt~R~
10
0K
10
x~t~KR~
K
02x22x2
0-12x22x202x22x2
10
0R~
10
x~K
10
t~R~
10
0KP 02x22x2
alternatives, because 8dof (3+3+2), not more
Summary parallel projection
1000
0010
0001
P canonical representation
10
0KK 22 calibration matrix
principal point is not defined
A hierarchy of affine cameras
Orthographic projection
Scaled orthographic projection
1000
0010
0001
P
10
tRH
10
r
r
P 22T
11T
t
t
k
t
t
/10
r
r
P 22T
11T
(5dof)
(6dof)
A hierarchy of affine cameras
Weak perspective projection (useful infinite appr. of unknown CCD camera)
k
t
t
y
x
/10
r
r
1
α
α
P 22T
11T
(7dof)
1. Affine camera=camera with principal plane coinciding with ∞
2. Affine camera maps parallel lines to parallel lines
3. No center of projection, but direction of projection PAD=0(point on ∞)
A hierarchy of affine cameras
Affine camera
k
t
ts
y
x
A
/10
r
r
1
α
α
P 22T
11T
(8dof)
1000P 2232221
1131211
tmmmtmmm
A
affine 44100000100001
affine 33P
A
Hierarchy of cameras
Camera center
Image plane Object plane
X0(origin)xpersp
Perspective:
xparap
Para-perspective:
First order approximation of perspective
z
y
x
persp
t
t
t
f
f
P
k
j
i
1
xorth
Orthographic:
1Ty
x
orth t
t
P
0
j
i
Weak perspective:
xwp
11 Ty
x
wp t
t
k
k
P
0
j
i
Other Cameras
Pushbroom cameras
T)(X X,Y,X,T T),,(PX wyx T)/,( wyx
Straight lines are not mapped to straight lines!
(otherwise it would be a projective camera)
(11dof)
Line cameras
ZYX
pppppp
yx
232221
131211
(5dof)
c~|IRKP 22222232
Null-space PC=0 yields camera centerAlso decomposition
• Projection equationProjection equation
xxii=P=PiiXX
• Resection:Resection:– xxii,X P,X Pii
A 3D Vision Problem: Multi-view geometry - resection
Given image points and 3D points calculate camera projection matrix.
Estimating camera matrix P
•Given a number of correspondences between 3-Given a number of correspondences between 3-
D points and their 2-D image projections D points and their 2-D image projections XXii x xii, we would like to determine the , we would like to determine the camera camera
projection matrixprojection matrix PP such that such that xxi i == PXPXii for for
all all ii
A Calibration Target
courtesy of B. Wilburn
XZ
Y
Xi
xi
Estimating P: The Direct Linear Transformation (DLT) Algorithm
• xxi i == PXPXii is an equation involving homogeneous is an equation involving homogeneous
vectors, so vectors, so PXPXii and and xxi i need only be in the same need only be in the same
direction, not strictly equaldirection, not strictly equal
• We can specify “same directionality” by using a We can specify “same directionality” by using a cross product formulation:cross product formulation:
DLT Camera Matrix Estimation: Preliminaries
•Let the image point Let the image point xxii = ( = (xxii, , yyii, , wwii))TT
(remember that (remember that XXii has 4 elements) has 4 elements)
•Denoting the Denoting the jjth row of th row of PP by by ppjTjT (a 4-element (a 4-element row vector), we have: row vector), we have:
DLT Camera Matrix Estimation: Step 1
•Then by the definition of the cross product, Then by the definition of the cross product,
xxi i PXPXii is: is:
DLT Camera Matrix Estimation: Step 2
•The dot product commutes, so The dot product commutes, so ppjTjT XXii ==
XXTTii ppjj, and we can rewrite the preceding as:, and we can rewrite the preceding as:
DLT Camera Matrix Estimation: Step 3
• Collecting terms, this can be rewritten as a Collecting terms, this can be rewritten as a matrix product:matrix product:
where where 00TT = (0, 0, 0, 0) = (0, 0, 0, 0). This is a . This is a 3 x 12 matrix times a 12-element column 3 x 12 matrix times a 12-element column vector vector pp = ( = (pp11TT, , pp22TT, , pp33TT))TT
What We Just Did
DLT Camera Matrix Estimation: Step 4
• There are only two linearly independent rows here There are only two linearly independent rows here – The third row is obtained by adding The third row is obtained by adding xxii times the first row to times the first row to yyii times the times the
second and scaling the sum by second and scaling the sum by -1/-1/wwii
DLT Camera Matrix Estimation: Step 4
• So we can eliminate one row to obtain the So we can eliminate one row to obtain the
following linear matrix equation for the following linear matrix equation for the iith pair th pair of corresponding points:of corresponding points:
• Write this as Write this as AAii pp = = 00
DLT Camera Matrix Estimation: Step 5
• Remember that there are 11 unknowns which Remember that there are 11 unknowns which generate the 3 x 4 homogeneous matrix generate the 3 x 4 homogeneous matrix PP
(represented in vector form by (represented in vector form by pp))• Each point correspondence yields 2 equations Each point correspondence yields 2 equations
(the two row of (the two row of AAii)) We need at least 5 ½ point correspondences to We need at least 5 ½ point correspondences to
solve for solve for pp• Stack Stack AAii to get homogeneous linear system to get homogeneous linear system AA p p
= 0= 0
Experiment: