3D photography Marc Pollefeys Fall 2008 http://www.inf.ethz.ch/personal/pomarc/courses/3dphoto

  • View
    212

  • Download
    0

Embed Size (px)

Transcript

  • Slide 1
  • 3D photography Marc Pollefeys Fall 2008 http://www.inf.ethz.ch/personal/pomarc/courses/3dphoto/
  • Slide 2
  • 3D Photography Obtaining 3D shape (and appearance) of real-world objects
  • Slide 3
  • Motivation Applications in many different areas A few examples
  • Slide 4
  • Surgery - simulation simulate results of surgery allows preoperative planning
  • Slide 5
  • Surgery - teaching Capture models of surgery for interactive learning
  • Slide 6
  • Clothing Scan a person, custom-fit clothing
  • Slide 7
  • Forensics Crime scene recording and analysis
  • Slide 8
  • Forensics
  • Slide 9
  • 3D urban modeling UNC/UKY UrbanScape project
  • Slide 10
  • Industrial inspection Verify specifications Compare measured model with CAD
  • Slide 11
  • Scanning industrial sites as-build 3D model of off-shore oil platform
  • Slide 12
  • Robot navigation small tethered rover pan/tilt stereo head ESA project our task: Calibration + Terrain modelling + Visualization
  • Slide 13
  • Architecture Survey Stability analysis Plan renovations
  • Slide 14
  • Architecture Survey Stability analysis Plan renovations
  • Slide 15
  • Cultural heritage Virtual Monticello Allow virtual visits
  • Slide 16
  • Cultural heritage Stanfords Digital Michelangelo Digital archive Art historic studies
  • Slide 17
  • Archaeology accuracy ~1/500 from DV video (i.e. 140kb jpegs 576x720)
  • Slide 18
  • Record different excavation layers Archaeology Generate & verify construction hypothesis Layer 1 Layer 2 Generate 4D excavation record
  • Slide 19
  • Computer games
  • Slide 20
  • Course objectives To understand the concepts that allow to recover 3D shape from images Explore the state of the art in 3D photography Implement a 3D photography system/algorithm
  • Slide 21
  • Material Slides and more http://www.inf.ethz.ch/personal/pomarc/courses/3dphoto/ Also check out on-line shape-from-video tutorial: http://www.cs.unc.edu/~marc/tutorial.pdf http://www.cs.unc.edu/~marc/tutorial/ Other interesting stuff: Book by Hartley & Zisserman, Multiple View Geometry Peter Allens 3D photography class webpage http://www1.cs.columbia.edu/~allen/F07/schedule.html http://www1.cs.columbia.edu/~allen/F07/schedule.html
  • Slide 22
  • Learning approach Lectures: cover main 3D photography concepts and approaches. Exercice sessions: Assignments (1 st half semester) Paper presentations and project updates (2 nd half semester) 3D photography project: Choose topic, define scope (1 st half semster) Implement algorithm/system (2 nd half semester) Grade distribution Assignments & paper presentation: 50% Course project: 50%
  • Slide 23
  • Content camera model and calibration single-view metrology triangulation epipolar geometry, stereo and rectification structured-light, active techniques feature tracking and matching structure-from-motion shape-from-silhouettes space-carving
  • Slide 24
  • 3D photography course schedule (tentative) LectureExercise Sept 17Introduction- Sept 24Geometry & Camera modelCamera calibration Oct. 1Single View MetrologyMeasuring in images Oct. 8Feature Tracking/matching (Jan-Michael Frahm?) Correspondence computation Oct. 15Epipolar Geometry (David Gallup?) F-matrix computation Oct. 22Shape-from-SilhouettesVisual-hull computation Oct. 29Stereo matchingProject proposals Nov. 5Structured light and active range sensing (TBD) Papers Nov. 12Structure from motionPapers Nov. 19Multi-view geometry and self-calibration Papers Nov. 26Shape-from-XPapers Dec. 33D modeling and registrationPapers Dec. 10Appearance modeling and image-based rendering (TBD) Papers Dec. 17Final project presentations
  • Slide 25
  • Fast Forward! Quick overview of what is coming
  • Slide 26
  • Camera models and geometry Pinhole camera Geometric transformations in 2D and 3D or
  • Slide 27
  • Camera Calibration Know 2D/3D correspondences, compute projection matrix also radial distortion (non-linear)
  • Slide 28
  • Single View Metrology
  • Slide 29
  • Antonio Criminisi
  • Slide 30
  • Feature tracking and matching Harris corners, KLT features, SIFT features key concepts: invariance of extraction, descriptors to viewpoint, exposure and illumination changes
  • Slide 31
  • l2l2 3D from images C1C1 m1m1 M? L1L1 m2m2 L2L2 M C2C2 Triangulation - calibration - correspondences
  • Slide 32
  • Epipolar Geometry Fundamental matrix Essential matrix Also how to robustly compute from images
  • Slide 33
  • Stereo and rectification Warp images to simplify epipolar geometry Compute correspondences for all pixels
  • Slide 34
  • Structured-light Projector = camera Use specific patterns to obtain correspondences
  • Slide 35
  • Initialize Motion (P 1,P 2 compatibel with F) Initialize Structure (minimize reprojection error) Extend motion (compute pose through matches seen in 2 or more previous views) Extend structure (Initialize new structure, refine existing structure) Structure from motion
  • Slide 36
  • ** ** projection constraints Auto-calibration
  • Slide 37
  • Shape-from-Silhouette
  • Slide 38
  • Space-carving Only keep photo-consistent voxels
  • Slide 39
  • Shape-from-X Shape-from-focus Shape-from-texture time-of-flight
  • Slide 40
  • 3D modeling and texturing Multiple depth imagesSurface model Texture integration patchwork texture map
  • Slide 41
  • Papers and discussion Will cover recent state of the art Each student will present a paper, discussion Select paper related to project
  • Slide 42
  • Course project: Build your own 3D scanner! Example: Bouguet ICCV98 Students can work in group or alone
  • Slide 43
  • 3D photography course team Marc Pollefeys CAB F66 marc.pollefeys@inf.ethz.edu Tel. 044 63 23105 David Gallup CAB F64.2 gallup@cs.unc.edu Tel. 044 23061