Text of Computer Vision Calibration Marc Pollefeys COMP 256 Many slides and illustrations from J. Ponce
Computer Vision Calibration Marc Pollefeys COMP 256 Many slides and illustrations from J. Ponce
Computer Vision Aug 26/28-Introduction Sep 2/4CamerasRadiometry Sep 9/11Sources & ShadowsColor Sep 16/18Linear filters & edges(hurricane Isabel) Sep 23/25Pyramids & TextureMulti-View Geometry Sep30/Oct2StereoProject proposals Oct 7/9Tracking (Welch)Optical flow Oct 14/16-- Oct 21/23Silhouettes/carving(Fall break) Oct 28/30-Structure from motion Nov 4/6Project updateProj. SfM Nov 11/13Camera calibrationSegmentation Nov 18/20FittingProb. segm.&fit. Nov 25/27Matching templates(Thanksgiving) Dec 2/4Matching relationsRange data Dec ?Final project Tentative class schedule
Computer Vision GEOMETRIC CAMERA MODELS Elements of Euclidean Geometry The Intrinsic Parameters of a Camera The Extrinsic Parameters of a Camera The General Form of the Perspective Projection Equation Line Geometry Reading: Chapter 2.
Computer Vision Quantitative Measurements and Calibration Euclidean Geometry
Computer Vision Euclidean Coordinate Systems
Computer Vision Planes
Computer Vision Coordinate Changes: Pure Translations O B P = O B O A + O A P, B P = A P + B O A
Computer Vision Coordinate Changes: Pure Rotations
Computer Vision Coordinate Changes: Rotations about the z Axis
Computer Vision A rotation matrix is characterized by the following properties: Its inverse is equal to its transpose, and its determinant is equal to 1. Or equivalently: Its rows (or columns) form a right-handed orthonormal coordinate system.
Computer Vision Coordinate Changes: Pure Rotations
Computer Vision Block Matrix Multiplication What is AB ? Homogeneous Representation of Rigid Transformations
Computer Vision Rigid Transformations as Mappings
Computer Vision Rigid Transformations as Mappings: Rotation about the k Axis
Computer Vision Pinhole Perspective Equation z y fy z x fx '' ''
Computer Vision The Intrinsic Parameters of a Camera Normalized Image Coordinates Physical Image Coordinates Units: k,l : pixel/m f : m : pixel
Computer Vision The Intrinsic Parameters of a Camera Calibration Matrix The Perspective Projection Equation
Computer Vision Extrinsic Parameters
Computer Vision Explicit Form of the Projection Matrix Note: M is only defined up to scale in this setting!!
Computer Vision Theorem (Faugeras, 1993)
Computer Vision GEOMETRIC CAMERA CALIBRATION The Calibration Problem Least-Squares Techniques Linear Calibration from Points Linear Calibration from Lines Analytical Photogrammetry Reading: Chapter 3
Computer Vision Calibration Problem
Computer Vision Linear Systems A A x xb b = = Square system: unique solution Gaussian elimination Rectangular system ?? underconstrained: infinity of solutions Minimize |Ax-b| 2 overconstrained: no solution
Computer Vision How do you solve overconstrained linear equations ??
Computer Vision Homogeneous Linear Systems A A x x0 0 = = Square system: unique solution: 0 unless Det(A)=0 Rectangular system ?? 0 is always a solution Minimize |Ax| under the constraint |x| =1 2 2
Computer Vision How do you solve overconstrained homogeneous linear equations ?? The solution is e. 1 remember: EIG(A T A)=SVD(A), i.e. solution is V n
Computer Vision Linear Camera Calibration
Computer Vision Once M is known, you still got to recover the intrinsic and extrinsic parameters !!! This is a decomposition problem, not an estimation problem. Intrinsic parameters Extrinsic parameters
Computer Vision Degenerate Point Configurations Are there other solutions besides M ?? Coplanar points: ( )=( ) or ( ) or ( ) Points lying on the intersection curve of two quadric surfaces = straight line + twisted cubic Does not happen for 6 or more random points!
Computer Vision Analytical Photogrammetry Non-Linear Least-Squares Methods Newton Gauss-Newton Levenberg-Marquardt Iterative, quadratically convergent in favorable situations
Computer Vision Mobile Robot Localization (Devy et al., 1997)
Computer Vision Absolute scale cannot be recovered! The Euclidean shape (defined up to an arbitrary similitude) is the best that can be recovered. From Projective to Euclidean Images If z, P, R and t are solutions, so are z, P, R and t.
Computer Vision From uncalibrated to calibrated cameras Perspective camera: Calibrated camera: Problem: what is Q ?
Computer Vision From uncalibrated to calibrated cameras II Perspective camera: Calibrated camera: Problem: what is Q ? Example: known image center
Computer Vision Sequential SfM Initialize motion from two images Initialize structure For each additional view Determine pose of camera Refine and extend structure Refine structure and motion
Computer Vision Initial projective camera motion Choose P and Pcompatible with F Reconstruction up to projective ambiguity (reference plane;arbitrary) Initialize motion Initialize structure For each additional view Determine pose of camera Refine and extend structure Refine structure and motion Same for more views? different projective basis
Computer Vision Initializing projective structure Reconstruct matches in projective frame by minimizing the reprojection error Non-iterative optimal solution Initialize motion Initialize structure For each additional view Determine pose of camera Refine and extend structure Refine structure and motion
Computer Vision Projective pose estimation Infere 2D-3D matches from 2D-2D matches Compute pose from (RANSAC,6pts) F X x Inliers: Initialize motion Initialize structure For each additional view Determine pose of camera Refine and extend structure Refine structure and motion
Computer Vision Refining structure Extending structure 2-view triangulation (Iterative linear) Initialize motion Initialize structure For each additional view Determine pose of camera Refine and extend structure Refine structure and motion Refining and extending structure
Computer Vision Refining structure and motion use bundle adjustment Also model radial distortion to avoid bias!
Computer Vision Metric structure and motion Note that a fundamental problem of the uncalibrated approach is that it fails if a purely planar scene is observed (in one or more views) (solution possible based on model selection) use self-calibration (see next class)
Computer Vision Dealing with dominant planes
Computer Vision PPPgric HHgric
Computer Vision Farmhouse 3D models (note: reconstruction much larger than camera field-of-view)
Computer Vision Application: video augmentation
Computer Vision Next class: Segmentation Reading: Chapter 14