Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
Prof. Dr. Daniel Cremers
Chair of Computer Vision & Artificial Intelligence
Departments of Informatics & Mathematics
Technical University of Munich
Computer Vision I: Variational Methods
Exercises
Emanuel LaudeMarvin Eisenberger
3Daniel Cremers Computer Vision I: Variational Methods
Computer Vision & Driver Assistance
4Daniel Cremers Computer Vision I: Variational Methods
Variational Scene Flow
Wedel et al. IJCV ‘11, Wedel & Cremers, Springer 2011
3300 delegates!
6Daniel Cremers Computer Vision I: Variational Methods
TUM Chair of Computer Vision & AI
7Daniel Cremers Computer Vision I: Variational Methods
Spatially Dense 3D Reconstruction
infinite-dimensional optimization
8Daniel Cremers Computer Vision I: Variational Methods
Which path is the fastest?
9Daniel Cremers Computer Vision I: Variational Methods
Bernoulli & The Brachistochrone
Johann Bernoulli (1667-1748)
10Daniel Cremers Computer Vision I: Variational Methods
Image segmentation:
Optimization in Computer Vision
Geman, Geman ’84, Blake, Zisserman ‘87, Kass et al. ’88,
Mumford, Shah ’89, Caselles et al. ‘95, Kichenassamy et al. ‘95,
Paragios, Deriche ’99, Chan, Vese ‘01, Tsai et al. ‘01, …
Multiview stereo reconstruction:
Faugeras, Keriven ’98, Duan et al. ‘04, Yezzi, Soatto ‘03,
Seitz et al. ‘06, Hernandez et al. ‘07, Labatut et al. ’07, …
Optical flow estimation:
Horn, Schunck ‘81, Nagel, Enkelmann ‘86, Black, Anandan ‘93,
Alvarez et al. ‘99, Brox et al. ‘04, Baker et al. ‘07, Zach et al. ‘07,
Sun et al. ‘08, Wedel et al. ’09, …
Non-convex energies
11Daniel Cremers Computer Vision I: Variational Methods
Non-convex energy Convex energy
Optimization and Convexity
12Daniel Cremers Computer Vision I: Variational Methods
Inverse Problems: Denoising
image denoised image
Rudin, Osher, Fatemi 1992, Goldlücke, Strekalovskiy, Cremers 2012
13Daniel Cremers Computer Vision I: Variational Methods
Leonhard Euler
(1703-1783)
Joseph-Louis Lagrange
(1736 – 1813)
Variational Methods
14Daniel Cremers Computer Vision I: Variational Methods
Variational Methods & PDEs
Euler-Lagrange equation as necessary condition:
Gradient descent:
15Daniel Cremers Computer Vision I: Variational Methods
Overview
Multiview reconstruction Stereo reconstruction
Realtime dense geometry
Super-res.textures
Reconstruction on the flyRGB-D cameras
16Daniel Cremers Computer Vision I: Variational Methods
Overview
Multiview reconstruction Stereo reconstruction
Realtime dense geometry
Super-res.textures
Reconstruction on the flyRGB-D cameras
17Daniel Cremers Computer Vision I: Variational Methods
3D Reconstruction from Multiple Views
Kolev, Klodt, Brox, Cremers, Int. J. of Computer Vision ’09:
Theorem: Globally optimal surfaces can be computed by convex optimization.
18Daniel Cremers Computer Vision I: Variational Methods
Evolution to Global Optimum
Kolev, Klodt, Brox, Cremers, Int. J. of Computer Vision ’09:
Theorem: Globally optimal surfaces can be computed by convex optimization.
19Daniel Cremers Computer Vision I: Variational Methods
Image data courtesy of Yasutaka Furukawa.
Reconstruction of Fine-scale Structures
20Daniel Cremers Computer Vision I: Variational Methods
Reconstructing the Niobids Statues
Kolev, Cremers, ECCV ’08, PAMI ‘11
21Daniel Cremers Computer Vision I: Variational Methods
Multiview Reconstruction
Kolev, Cremers, ECCV ’08, PAMI ‘11
22Daniel Cremers Computer Vision I: Variational Methods
Reconstructing Dynamic Scenes
Oswald, Stühmer, Cremers, ECCV ‘14
23Daniel Cremers Computer Vision I: Variational Methods
Action Reconstruction
Oswald, Stühmer, Cremers, ECCV ‘14
24Daniel Cremers Computer Vision I: Variational Methods
Single View Reconstruction
Can we recover geometry from a single image?
Yes: Shape-from-shading, shape-from-focus, shape from symmetry,…
Solution: Fixed-volume silhouette-consistent minimal surface.
25Daniel Cremers Computer Vision I: Variational Methods
Toeppe, Oswald, Rother, Cremers, ACCV 2010
Single View Reconstruction
26Daniel Cremers Computer Vision I: Variational Methods
Toeppe et al. ACCV 2010, Oswald et al. CVPR 2012
Single View Reconstruction
Reconstruction computed in fractions of a second on GPU
27Daniel Cremers Computer Vision I: Variational Methods
Toeppe, Oswald, Rother, Cremers, ACCV 2010
Single View Reconstruction
28Daniel Cremers Computer Vision I: Variational Methods
Modifying the Material Properties
29Daniel Cremers Computer Vision I: Variational Methods
Toeppe, Oswald, Rother, Cremers, ACCV 2010
Single View Reconstruction
30Daniel Cremers Computer Vision I: Variational Methods
Toeppe, Oswald, Rother, Cremers, ACCV 2010*
In collaboration with Microsoft Research
* Best Paper Honorable Mention
Single View Reconstruction
31Daniel Cremers Computer Vision I: Variational Methods
Overview
Multiview reconstruction Stereo reconstruction
Realtime dense geometry
Super-res.textures
Reconstruction on the flyRGB-D cameras
32Daniel Cremers Computer Vision I: Variational Methods
Evolution to Global Optimum
Kolev, Klodt, Brox, Cremers, IJCV 2009
33Daniel Cremers Computer Vision I: Variational Methods
Super-Resolution Texture Map
Given all images determine the surface color
back-projectionblur & downsample
Goldlücke, Cremers, ICCV ’09, DAGM ’09*, IJCV ‘13* Best Paper
Award
34Daniel Cremers Computer Vision I: Variational Methods
Super-Resolution Texture Map
Goldlücke, Cremers, ICCV ’09, DAGM ’09*, IJCV ‘13* Best Paper
Award
35Daniel Cremers Computer Vision I: Variational Methods
Weighted average Super-resolution texture
Super-Resolution Texture Map
Goldlücke, Cremers, ICCV ’09, DAGM ’09*, IJCV ‘13* Best Paper
Award
36Daniel Cremers Computer Vision I: Variational Methods
Closeup of input image Super-resolution texture
Super-Resolution Texture Map
Goldlücke, Cremers, ICCV ’09, DAGM ’09*, IJCV ‘13* Best Paper
Award
37Daniel Cremers Computer Vision I: Variational Methods
Overview
Multiview reconstruction Stereo reconstruction
Realtime dense geometry
Super-res.textures
Reconstruction on the flyRGB-D cameras
38Daniel Cremers Computer Vision I: Variational Methods
Example: Stereo Reconstruction
From Binary to Multilabel Optimization
Pock, Schoenemann, Bischof, Cremers, Europ. Conf. on Computer Vision ’08:
Theorem: Stereo reconstruction can be solved by convex optimization.
39Daniel Cremers Computer Vision I: Variational Methods
Evolution to Global Minimum
40Daniel Cremers Computer Vision I: Variational Methods
1/2 input images (6 Mpixel)
Courtesy of H. Hirschmüller
Depth reconstruction
77 seconds
Reconstruction from Aerial Images
Stangl, Souiai, Cremers, GCPR ‘13
41Daniel Cremers Computer Vision I: Variational Methods
One of two input imagesDepth reconstruction
Courtesy of Microsoft
Reconstruction from Aerial Images
42Daniel Cremers Computer Vision I: Variational Methods
Reconstruction from Aerial Images
43Daniel Cremers Computer Vision I: Variational Methods
Highly accurate Stereo Reconstruction
Möllenhoff, Laude, Möller, Lellmann, Cremers, CVPR ’16 *
* Best Paper Honorable Mention
Stereo input Depth reconstruction
44Daniel Cremers Computer Vision I: Variational Methods
1/2 input images (1000x1000) Depth reconstruction
Munich from the Air
Kuschk, Cremers, ICCV Big Data Workshop 2013
45Daniel Cremers Computer Vision I: Variational Methods
Overview
Multiview reconstruction Stereo reconstruction
Realtime dense geometry
Super-res.textures
Reconstruction on the flyRGB-D cameras
46Daniel Cremers Computer Vision I: Variational Methods
Input video Optical flow field
From Dense Flow to Dense Geometry
Horn & Schunck ‘81, Zach et al. DAGM ’07, Wedel et al. ICCV ’09
47Daniel Cremers Computer Vision I: Variational Methods
*
* 60 fps @ 640x480
Horn & Schunck ‘81, Zach et al. DAGM ’07, Wedel et al. ICCV ’09
Input video Optical flow field
From Dense Flow to Dense Geometry
48Daniel Cremers Computer Vision I: Variational Methods
Dense geometry from hand-held camera
Stuehmer , Gumhold, Cremers, DAGM ’10
Brightness constancy:
49Daniel Cremers Computer Vision I: Variational Methods
Dense geometry from hand-held camera
Stuehmer, Gumhold, Cremers, DAGM ’10
50Daniel Cremers Computer Vision I: Variational Methods
Dense geometry from hand-held camera
Stuehmer, Gumhold, Cremers, DAGM ’10
51Daniel Cremers Computer Vision I: Variational Methods
16.0 fps 22.0 fps 41.1 fps
Stuehmer, Gumhold, Cremers, DAGM ’10
Realtime Dense Reconstruction
52Daniel Cremers Computer Vision I: Variational Methods
Overview
Multiview reconstruction Stereo reconstruction
Realtime dense geometry
Super-res.textures
Reconstruction on the flyRGB-D cameras
53Daniel Cremers Computer Vision I: Variational Methods
RGB-D Camera Tracking
Steinbruecker et al. ICCV ’11, Kerl et al., ICRA ‘13
Optimize dense photo-consistency:
54Daniel Cremers Computer Vision I: Variational Methods
Realtime 3D Modeling
Color input Depth input
55Daniel Cremers Computer Vision I: Variational Methods
Realtime 3D Modeling
56Daniel Cremers Computer Vision I: Variational Methods
Realtime 3D Modeling
57Daniel Cremers Computer Vision I: Variational Methods
Realtime 3D Modeling
58Daniel Cremers Computer Vision I: Variational Methods
Realtime 3D Modeling
59Daniel Cremers Computer Vision I: Variational Methods
Realtime 3D Modeling
60Daniel Cremers Computer Vision I: Variational Methods
Realtime 3D Modeling
61Daniel Cremers Computer Vision I: Variational Methods
Realtime 3D Modeling
62Daniel Cremers Computer Vision I: Variational Methods
Full-Body Scanner
63Daniel Cremers Computer Vision I: Variational Methods
Overview
Multiview reconstruction Stereo reconstruction
Realtime dense geometry
Super-res.textures
Reconstruction on the flyRGB-D cameras
64Daniel Cremers Computer Vision I: Variational Methods
Reconstruction on the Fly
Bylow, Sturm, Kerl, Kahl, Cremers RSS ‘13
65Daniel Cremers Computer Vision I: Variational Methods
Kerl, Sturm, Cremers ICRA ‘13
Large Scale: Loop Closure
66Daniel Cremers Computer Vision I: Variational Methods
Large Scale: Octrees
Steinbrücker, Kerl, Sturm, Cremers ICCV ‘13
67Daniel Cremers Computer Vision I: Variational Methods
Realtime Large-Scale Reconstruction
Steinbrücker, Kerl, Sturm, Cremers ICCV ‘13, ICRA ‘14
68Daniel Cremers Computer Vision I: Variational Methods
Summary
multiview reconstruction super-res. textures action reconstruction
3D on the flyRGB-D modelingstereo reconstruction