Robot navigation
small tethered rover
pan/tilt stereo head
ESA projectour task: Calibration + Terrain modelling + Visualization
Record different excavation layers
Archaeology
Generate & verify construction hypothesis
Layer 1 Layer 2
Generate 4D excavation record
Course objectives
• To understand the concepts that allow to recover 3D shape from images
• Explore the state of the art in 3D photography
• Implement a 3D photography system/algorithm
Material
Slides and more
http://www.inf.ethz.ch/personal/pomarc/courses/3dphoto/
Also check out on-line “shape-from-video” tutorial:http://www.cs.unc.edu/~marc/tutorial.pdfhttp://www.cs.unc.edu/~marc/tutorial/
Other interesting stuff:
• Book by Hartley & Zisserman, Multiple View Geometry
• Peter Allen’s 3D photography class webpage http://www1.cs.columbia.edu/~allen/F07/schedule.html
Learning approach
• Lectures: • cover main 3D photography concepts and
approaches.• Exercice sessions:
• Assignments (1st half semester) • Paper presentations and project updates (2nd
half semester) • 3D photography project:
• Choose topic, define scope (1st half semster)• Implement algorithm/system (2nd half semester)
Grade distribution
• Assignments & paper presentation: 50%• Course project: 50%
Content
• camera model and calibration• single-view metrology• triangulation• epipolar geometry, stereo and
rectification• structured-light, active techniques• feature tracking and matching• structure-from-motion• shape-from-silhouettes• space-carving• …
3D photography course schedule(tentative)
Lecture Exercise
Sept 17 Introduction -
Sept 24 Geometry & Camera model Camera calibration
Oct. 1 Single View Metrology Measuring in images
Oct. 8 Feature Tracking/matching(Jan-Michael Frahm?)
Correspondence computation
Oct. 15 Epipolar Geometry(David Gallup?)
F-matrix computation
Oct. 22 Shape-from-Silhouettes Visual-hull computation
Oct. 29 Stereo matching Project proposals
Nov. 5 Structured light and active range sensing
(TBD)
Papers
Nov. 12 Structure from motion Papers
Nov. 19 Multi-view geometry and self-calibration
Papers
Nov. 26 Shape-from-X Papers
Dec. 3 3D modeling and registration Papers
Dec. 10 Appearance modeling and image-based rendering
(TBD)
Papers
Dec. 17 Final project presentations
Camera Calibration
• Know 2D/3D correspondences, compute projection matrix
also radial distortion (non-linear)
Feature tracking and matching
Harris corners, KLT features, SIFT featureskey concepts: invariance of extraction, descriptors to viewpoint, exposure and illumination changes
Stereo and rectificationWarp images to simplify epipolar geometry
Compute correspondences for all pixels
Initialize Motion (P1,P2 compatibel with F)
Initialize Structure (minimize reprojection error)
Extend motion(compute pose through matches seen in 2 or more previous views)
Extend structure(Initialize new structure, refine existing structure)
Structure from motion
3D modeling and texturing
Multiple depth images Surface model
Texture integration
patchwork texture map
Papers and discussion
• Will cover recent state of the art
Each student will present a paper, discussion
Select paper related to project
Course project: Build your own 3D scanner!
Example: Bouguet ICCV’98
Students can work in group or alone
3D photography course team
Marc PollefeysCAB [email protected]. 044 63 23105
David GallupCAB [email protected]. 044 23061