- Home
- Documents
*Multiple View Reconstruction Class 24 Multiple View Geometry Comp 290-089 Marc Pollefeys*

prev

next

of 32

View

213Download

0

Embed Size (px)

- Slide 1
- Multiple View Reconstruction Class 24 Multiple View Geometry Comp 290-089 Marc Pollefeys
- Slide 2
- Content Background: Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation. Single View: Camera model, Calibration, Single View Geometry. Two Views: Epipolar Geometry, 3D reconstruction, Computing F, Computing structure, Plane and homographies. Three Views: Trifocal Tensor, Computing T. More Views: N-Linearities, Self-Calibration,Multi View Reconstruction, Bundle adjustment, Dynamic SfM, Cheirality, Duality
- Slide 3
- Multi-view computation
- Slide 4
- practical structure and motion recovery from images Obtain reliable matches using matching or tracking and 2/3-view relations Compute initial structure and motion sequential structure and motion recovery hierarchical structure and motion recovery Refine structure and motion bundle adjustment Auto-calibrate Refine metric structure and motion
- Slide 5
- Sequential structure and motion recovery Initialize structure and motion from 2 views For each additional view Determine pose Refine and extend structure
- Slide 6
- Initial structure and motion Epipolar geometry Projective calibration compatible with F Yields correct projective camera setup (Faugeras92,Hartley92) Obtain structure through triangulation Use reprojection error for minimization Avoid measurements in projective space
- Slide 7
- Compute P i+1 using robust approach (6-point RANSAC) Extend and refine reconstruction 2D-2D 2D-3D mimi m i+1 M new view Determine pose towards existing structure
- Slide 8
- Non-sequential image collections 4.8im/pt 64 images 3792 points Problem: Features are lost and reinitialized as new features Solution: Match with other close views
- Slide 9
- For every view i Extract features Compute two view geometry i-1/i and matches Compute pose using robust algorithm Refine existing structure Initialize new structure Relating to more views Problem: find close views in projective frame For every view i Extract features Compute two view geometry i-1/i and matches Compute pose using robust algorithm For all close views k Compute two view geometry k/i and matches Infer new 2D-3D matches and add to list Refine pose using all 2D-3D matches Refine existing structure Initialize new structure
- Slide 10
- Refining and extending structure Refining structure Extending structure Triangulation (Iterative linear) (Hartley&Sturm,CVIU97) Initialize motion Initialize structure For each additional view Determine pose of camera Refine and extend structure Refine structure and motion
- Slide 11
- Structure and motion: example 190 images 7000points Input sequence Viewpoint surface mesh calibration demo
- Slide 12
- ULM demo
- Slide 13
- Hierarchical structure and motion recovery Compute 2-view Compute 3-view Stitch 3-view reconstructions Merge and refine reconstruction F T H PM
- Slide 14
- Stitching 3-view reconstructions Different possibilities 1. Align (P 2,P 3 ) with (P 1,P 2 ) 2. Align X,X (and CC) 3. Minimize reproj. error 4. MLE (merge)
- Slide 15
- Refining structure and motion Minimize reprojection error Maximum Likelyhood Estimation (if error zero-mean Gaussian noise) Huge problem but can be solved efficiently (Bundle adjustment)
- Slide 16
- Non-linear least-squares Newton iteration Levenberg-Marquardt Sparse Levenberg-Marquardt
- Slide 17
- Newton iteration Taylor approximation Jacobian normal eq.
- Slide 18
- Levenberg-Marquardt Augmented normal equations Normal equations solve again accept small ~ Newton (quadratic convergence) large ~ descent (guaranteed decrease)
- Slide 19
- Levenberg-Marquardt Requirements for minimization Function to compute f Start value P 0 Optionally, function to compute J (but numerical derivation ok too)
- Slide 20
- Sparse Levenberg-Marquardt complexity for solving prohibitive for large problems (100 views 10,000 points ~30,000 unknowns) Partition parameters partition A partition B (only dependent on A and itself) typically A contains camera parameters, and B contains 3D points
- Slide 21
- Sparse bundle adjustment residuals: normal equations: with
- Slide 22
- Sparse bundle adjustment normal equations: modified normal equations: solve in two parts:
- Slide 23
- Sparse bundle adjustment U1U1 U2U2 U3U3 WTWT W V P1P1 P2P2 P3P3 M Jacobian of has sparse block structure 12xm 3xn (in general much larger) im.pts. view 1 Needed for non-linear minimization
- Slide 24
- Sparse bundle adjustment Eliminate dependence of camera/motion parameters on structure parameters Note in general 3n >> 11m WTWT V U-WV -1 W T 11xm 3xn Allows much more efficient computations e.g. 100 views,10000 points, solve 1000x1000, not 30000x30000 Often still band diagonal use sparse linear algebra algorithms
- Slide 25
- Sparse bundle adjustment normal equations: modified normal equations: solve in two parts:
- Slide 26
- Sparse bundle adjustment Covariance estimation
- Slide 27
- Degenerate configurations Camera resectioning Two views More views (H&Z Ch.21)
- Slide 28
- Camera resectioning Cameras as points 2D case Chasles theorem
- Slide 29
- Ambiguity for 3D cameras Twisted cubic (or less) meeting lin. subspace(s) (degree+dimension