Computer Vision Calibration Marc Pollefeys COMP 256 Many slides and illustrations from J. Ponce

Embed Size (px)

Text of Computer Vision Calibration Marc Pollefeys COMP 256 Many slides and illustrations from J. Ponce

  • Slide 1
  • Computer Vision Calibration Marc Pollefeys COMP 256 Many slides and illustrations from J. Ponce
  • Slide 2
  • Computer Vision Aug 26/28-Introduction Sep 2/4CamerasRadiometry Sep 9/11Sources & ShadowsColor Sep 16/18Linear filters & edges(hurricane Isabel) Sep 23/25Pyramids & TextureMulti-View Geometry Sep30/Oct2StereoProject proposals Oct 7/9Tracking (Welch)Optical flow Oct 14/16-- Oct 21/23Silhouettes/carving(Fall break) Oct 28/30-Structure from motion Nov 4/6Project updateProj. SfM Nov 11/13Camera calibrationSegmentation Nov 18/20FittingProb. segm.&fit. Nov 25/27Matching templates(Thanksgiving) Dec 2/4Matching relationsRange data Dec ?Final project Tentative class schedule
  • Slide 3
  • Computer Vision GEOMETRIC CAMERA MODELS Elements of Euclidean Geometry The Intrinsic Parameters of a Camera The Extrinsic Parameters of a Camera The General Form of the Perspective Projection Equation Line Geometry Reading: Chapter 2.
  • Slide 4
  • Computer Vision Quantitative Measurements and Calibration Euclidean Geometry
  • Slide 5
  • Computer Vision Euclidean Coordinate Systems
  • Slide 6
  • Computer Vision Planes
  • Slide 7
  • Computer Vision Coordinate Changes: Pure Translations O B P = O B O A + O A P, B P = A P + B O A
  • Slide 8
  • Computer Vision Coordinate Changes: Pure Rotations
  • Slide 9
  • Computer Vision Coordinate Changes: Rotations about the z Axis
  • Slide 10
  • Computer Vision A rotation matrix is characterized by the following properties: Its inverse is equal to its transpose, and its determinant is equal to 1. Or equivalently: Its rows (or columns) form a right-handed orthonormal coordinate system.
  • Slide 11
  • Computer Vision Coordinate Changes: Pure Rotations
  • Slide 12
  • Computer Vision Coordinate Changes: Rigid Transformations
  • Slide 13
  • Computer Vision Block Matrix Multiplication What is AB ? Homogeneous Representation of Rigid Transformations
  • Slide 14
  • Computer Vision Rigid Transformations as Mappings
  • Slide 15
  • Computer Vision Rigid Transformations as Mappings: Rotation about the k Axis
  • Slide 16
  • Computer Vision Pinhole Perspective Equation z y fy z x fx '' ''
  • Slide 17
  • Computer Vision The Intrinsic Parameters of a Camera Normalized Image Coordinates Physical Image Coordinates Units: k,l : pixel/m f : m : pixel
  • Slide 18
  • Computer Vision The Intrinsic Parameters of a Camera Calibration Matrix The Perspective Projection Equation
  • Slide 19
  • Computer Vision Extrinsic Parameters
  • Slide 20
  • Computer Vision Explicit Form of the Projection Matrix Note: M is only defined up to scale in this setting!!
  • Slide 21
  • Computer Vision Theorem (Faugeras, 1993)
  • Slide 22
  • Computer Vision GEOMETRIC CAMERA CALIBRATION The Calibration Problem Least-Squares Techniques Linear Calibration from Points Linear Calibration from Lines Analytical Photogrammetry Reading: Chapter 3
  • Slide 23
  • Computer Vision Calibration Problem
  • Slide 24
  • Computer Vision Linear Systems A A x xb b = = Square system: unique solution Gaussian elimination Rectangular system ?? underconstrained: infinity of solutions Minimize |Ax-b| 2 overconstrained: no solution
  • Slide 25
  • Computer Vision How do you solve overconstrained linear equations ??
  • Slide 26
  • Computer Vision Homogeneous Linear Systems A A x x0 0 = = Square system: unique solution: 0 unless Det(A)=0 Rectangular system ?? 0 is always a solution Minimize |Ax| under the constraint |x| =1 2 2
  • Slide 27
  • Computer Vision How do you solve overconstrained homogeneous linear equations ?? The solution is e. 1 remember: EIG(A T A)=SVD(A), i.e. solution is V n
  • Slide 28
  • Computer Vision Linear Camera Calibration
  • Slide 29
  • Computer Vision Once M is known, you still got to recover the intrinsic and extrinsic parameters !!! This is a decomposition problem, not an estimation problem. Intrinsic parameters Extrinsic parameters
  • Slide 30
  • Computer Vision Degenerate Point Configurations Are there other solutions besides M ?? Coplanar points: ( )=( ) or ( ) or ( ) Points lying on the intersection curve of two quadric surfaces = straight line + twisted cubic Does not happen for 6 or more random points!
  • Slide 31
  • Computer Vision Analytical Photogrammetry Non-Linear Least-Squares Methods Newton Gauss-Newton Levenberg-Marquardt Iterative, quadratically convergent in favorable situations
  • Slide 32
  • Computer Vision Mobile Robot Localization (Devy et al., 1997)
  • Slide 33
  • Computer Vision Absolute scale cannot be recovered! The Euclidean shape (defined up to an arbitrary similitude) is the best that can be recovered. From Projective to Euclidean Images If z, P, R and t are solutions, so are z, P, R and t.
  • Slide 34
  • Computer Vision From uncalibrated to calibrated cameras Perspective camera: Calibrated camera: Problem: what is Q ?
  • Slide 35
  • Computer Vision From uncalibrated to calibrated cameras II Perspective camera: Calibrated camera: Problem: what is Q ? Example: known image center
  • Slide 36
  • Computer Vision Sequential SfM Initialize motion from two images Initialize structure For each additional view Determine pose of camera Refine and extend structure Refine structure and motion
  • Slide 37
  • Computer Vision Initial projective camera motion Choose P and Pcompatible with F Reconstruction up to projective ambiguity (reference plane;arbitrary) Initialize motion Initialize structure For each additional view Determine pose of camera Refine and extend structure Refine structure and motion Same for more views? different projective basis
  • Slide 38
  • Computer Vision Initializing projective structure Reconstruct matches in projective frame by minimizing the reprojection error Non-iterative optimal solution Initialize motion Initialize structure For each additional view Determine pose of camera Refine and extend structure Refine structure and motion
  • Slide 39
  • Computer Vision Projective pose estimation Infere 2D-3D matches from 2D-2D matches Compute pose from (RANSAC,6pts) F X x Inliers: Initialize motion Initialize structure For each additional view Determine pose of camera Refine and extend structure Refine structure and motion
  • Slide 40
  • Computer Vision Refining structure Extending structure 2-view triangulation (Iterative linear) Initialize motion Initialize structure For each additional view Determine pose of camera Refine and extend structure Refine structure and motion Refining and extending structure
  • Slide 41
  • Computer Vision Refining structure and motion use bundle adjustment Also model radial distortion to avoid bias!
  • Slide 42
  • Computer Vision Metric structure and motion Note that a fundamental problem of the uncalibrated approach is that it fails if a purely planar scene is observed (in one or more views) (solution possible based on model selection) use self-calibration (see next class)
  • Slide 43
  • Computer Vision Dealing with dominant planes
  • Slide 44
  • Computer Vision PPPgric HHgric
  • Slide 45
  • Computer Vision Farmhouse 3D models (note: reconstruction much larger than camera field-of-view)
  • Slide 46
  • Computer Vision Application: video augmentation
  • Slide 47
  • Computer Vision Next class: Segmentation Reading: Chapter 14