View
218
Download
0
Embed Size (px)
Citation preview
ComputerVision
Calibration
Marc PollefeysCOMP 256
Many slides and illustrations from J. Ponce
ComputerVision
Aug 26/28 - Introduction
Sep 2/4 Cameras Radiometry
Sep 9/11 Sources & Shadows Color
Sep 16/18 Linear filters & edges
(hurricane Isabel)
Sep 23/25 Pyramids & Texture Multi-View Geometry
Sep30/Oct2 Stereo Project proposals
Oct 7/9 Tracking (Welch) Optical flow
Oct 14/16 - -
Oct 21/23 Silhouettes/carving (Fall break)
Oct 28/30 - Structure from motion
Nov 4/6 Project update Proj. SfM
Nov 11/13 Camera calibration Segmentation
Nov 18/20 Fitting Prob. segm.&fit.
Nov 25/27 Matching templates (Thanksgiving)
Dec 2/4 Matching relations Range data
Dec ? Final project
Tentative class schedule
ComputerVision GEOMETRIC CAMERA MODELS
• Elements of Euclidean Geometry
• The Intrinsic Parameters of a Camera
• The Extrinsic Parameters of a Camera
• The General Form of the Perspective Projection Equation
• Line Geometry
Reading: Chapter 2.
ComputerVision Quantitative Measurements and Calibration
Euclidean Geometry
ComputerVision Euclidean Coordinate Systems
z
y
x
zyxOP
OPz
OPy
OPx
Pkji
k
j
i
.
.
.
ComputerVision Planes
1
and where
0.00.
zyx
dcba
dczbyaxAP
PΠ
PΠn
ComputerVision Coordinate Changes: Pure Translations
OBP = OBOA + OAP , BP = AP + BOA
ComputerVision
Coordinate Changes: Pure Rotations
BABABA
BABABA
BABABABA R
kkkjki
jkjjji
ikijii
...
...
...
AB
AB
AB kji
TB
A
TB
A
TB
A
k
j
i
ComputerVision
Coordinate Changes: Rotations about the z Axis
100
0cossin
0sincos
RBA
ComputerVision A rotation matrix is characterized by the following properties:
• Its inverse is equal to its transpose, and
• its determinant is equal to 1.
Or equivalently:
• Its rows (or columns) form a right-handedorthonormal coordinate system.
ComputerVision Coordinate Changes:
Pure Rotations
PRP
z
y
x
z
y
x
OP
ABA
B
B
B
B
BBBA
A
A
AAA
kjikji
ComputerVision Coordinate Changes: Rigid Transformations
ABAB
AB OPRP
ComputerVision
Block Matrix Multiplication
2221
1211
2221
1211
BB
BBB
AA
AAA
What is AB ?
2222122121221121
2212121121121111
BABABABA
BABABABAAB
Homogeneous Representation of Rigid Transformations
11111
PT
OPRPORP ABA
ABAB
AA
TA
BBA
B
0
ComputerVision Rigid Transformations as Mappings
ComputerVision
Rigid Transformations as Mappings: Rotation about the k Axis
ComputerVision Pinhole Perspective Equation
z
yfy
z
xfx
''
''
ComputerVision
The Intrinsic Parameters of a Camera
Normalized ImageCoordinates
Physical Image Coordinates
Units:
k,l : pixel/m
f : m
: pixel
ComputerVision
The Intrinsic Parameters of a Camera
Calibration Matrix
The PerspectiveProjection Equation
1
0 .. 2
2h
w
ff
Kge
ComputerVision
Extrinsic Parameters
ComputerVision
Explicit Form of the Projection Matrix
Note:
M is only defined up to scale in this setting!!
ComputerVision
Theorem (Faugeras, 1993)
ComputerVision GEOMETRIC CAMERA CALIBRATION
• The Calibration Problem
• Least-Squares Techniques
• Linear Calibration from Points
• Linear Calibration from Lines
• Analytical Photogrammetry
Reading: Chapter 3
ComputerVision
Calibration Problem
ComputerVision Linear Systems
A
A
x
x b
b=
=
Square system:
• unique solution
• Gaussian elimination
Rectangular system ??
• underconstrained: infinity of solutions
Minimize |Ax-b|2
• overconstrained: no solution
ComputerVision
How do you solve overconstrained linear equations ??
ComputerVision
Homogeneous Linear Systems
A
A
x
x 0
0=
=
Square system:
• unique solution: 0
• unless Det(A)=0
Rectangular system ??
• 0 is always a solution
Minimize |Ax| under the constraint |x| =12
2
ComputerVision
How do you solve overconstrained homogeneous linear equations ??
The solution is e .1
remember: EIG(ATA)=SVD(A), i.e. solution is Vn
ComputerVision Linear Camera Calibration
ComputerVision Once M is known, you still got to recover the intrinsic and
extrinsic parameters !!!
This is a decomposition problem, not an estimationproblem.
• Intrinsic parameters
• Extrinsic parameters
ComputerVision
Degenerate Point Configurations
Are there other solutions besides M ??
• Coplanar points: ()=() or () or ()
• Points lying on the intersection curve of two quadricsurfaces = straight line + twisted cubic
Does not happen for 6 or more random points!
ComputerVision
Analytical Photogrammetry
Non-Linear Least-Squares Methods
• Newton• Gauss-Newton• Levenberg-Marquardt
Iterative, quadratically convergent in favorable situations
ComputerVision Mobile Robot Localization (Devy et al., 1997)
ComputerVision
Absolute scale cannot be recovered! The Euclidean shape(defined up to an arbitrary similitude) is the best that can berecovered.
From Projective to Euclidean Images
If z , P , R and t are solutions, so are z , P , R and t .
ComputerVision From uncalibrated to calibrated cameras
Perspective camera:
Calibrated camera:
Problem: what is Q ?
ComputerVision From uncalibrated to calibrated cameras II
Perspective camera:
Calibrated camera:
Problem: what is Q ?
Example: known image center
ComputerVision Sequential SfM
• Initialize motion from two images• Initialize structure• For each additional view
– Determine pose of camera– Refine and extend structure
• Refine structure and motion
ComputerVision
Initial projective camera motion
• Choose P and P´compatible with F
Reconstruction up to projective ambiguity
(reference plane;arbitrary)
•Initialize motion•Initialize structure•For each additional view
•Determine pose of camera•Refine and extend structure
•Refine structure and motion
Same for more views?
different projective basis
ComputerVision Initializing projective structure
• Reconstruct matches in projective frame by minimizing the reprojection error
Non-iterative optimal solution •Initialize motion•Initialize structure•For each additional view
•Determine pose of camera•Refine and extend structure
•Refine structure and motion
ComputerVision Projective pose estimation
• Infere 2D-3D matches from 2D-2D matches
• Compute pose from (RANSAC,6pts)
F
X
x
Inliers: inx,X x X DD iii P
•Initialize motion•Initialize structure•For each additional view
•Determine pose of camera•Refine and extend structure
•Refine structure and motion
ComputerVision
• Refining structure
• Extending structure2-view triangulation
X~
P
1
3
(Iterative linear)
•Initialize motion•Initialize structure•For each additional view
•Determine pose of camera•Refine and extend structure
•Refine structure and motion
Refining and extending structure
ComputerVision Refining structure and motion
• use bundle adjustment
Also model radial distortion to avoid bias!
ComputerVision Metric structure and motion
Note that a fundamental problem of the uncalibrated approach is that it fails if a purely planar scene is observed (in one or more views)
(solution possible based on model selection)
use self-calibration (see next class)
ComputerVision Dealing with dominant planes
ComputerVision
PPPgric
HHgric
ComputerVision Farmhouse 3D models
(note: reconstruction much larger than camera field-of-view)
ComputerVision Application: video augmentation
ComputerVision Next class:
Segmentation
Reading: Chapter 14