View
220
Download
0
Category
Preview:
Citation preview
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 1
Computational Plenoptic ImagingGordon Wetzstein1 Ivo Ihrke2 Douglas Lanman3 Wolfgang Heidrich1
1University of British Columbia 2Saarland University 3MIT Media Lab
Eurographics 2011 – State of the Art Report
VI. Multiplexing Time
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 2
History – Eadweard Muybridge 1830-1904
http://en.wikipedia.org/wiki/Eadweard_Muybridge
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 3
History – Étienne-Jules Marey 1830-1904
www.wikipedia.org
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 4
VI.I Time Lapse Photography
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 5
BBC Time Lapse – Look It!
ND 3.0 filter, f22, 1 minute exposure
• Long exposures to avoid temporal aliasing
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 6
VI.II High-Speed Imaging
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 7
High-Speed Cameras
Vision Research Phantom Flex (CMOS)2570 fps at HD resolution
Photron FASTCAM SA5 (CMOS)7500 fps at megapixel resolutionone million fps at 64x64 pixels
Casio Exilim Series (consumer cam)1000 fps at reduced resolution
Shimadzu HyperVision HPV-2 (CCD)one million fps at 312x260 pixels
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 8
Assorted Pixels
[Narasimhan & Nayar 05]
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 9
Temporal Mosaic with DMD
[Bub et al. 10]
DMD aligned with CCD in microscope
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 10
Non-Destructive Sensor Readout & Pixim
Cypress Semiconductor LUPA 30003 megapixels, 485 fps
www.pixim.com
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 11
Coded Rolling Shutter
[Gu et al. 10]
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 12
Reinterpretable Imager
• Moving pinhole over time in aperture
• Capture with light field camera
[Agrawal et al. 10]
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 13
Bullet Time Effect
from ‘The Matrix’
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 14
Stanford Multi-Camera Array
[Wilburn et al. 04]
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 15
Coded Temporal Sampling
[Agrawal et al. 10]
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 16
High-Speed Illumination – Electronic Strobes
Harold ‘Doc’ Edgerton1903-1990
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 17
Temporal Dithering with DLP Illumination
[Narasimhan et al. 08]
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 18
Coded Strobing Photography
[Reddy et al. 11]
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 19
Streak Cameras
www.hamamatsu.com
C5680$200K
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 20
VI.I Motion Deblurring
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 21
Motion Deblurring Overview
• Motion blur is velocity-dependent
• Can be described as convolution, where– Kernel shape is motion trajectory– Trajectory is modulated by exposure function
http://en.wikipedia.org/wiki/Motion_blur
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 22
Deconvolution is Still Hard
• Again – problems:
– Camera noise
– Spatially varying kernel (velocity-dependent)
– Unknown motion trajectory
– Ill-posed problem, kernel of box integration function is not invertible (optical cancellation of image frequencies)
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 23
Approaches to Improve Motion Deblurring
• Make PSF invertible coded exposure
• Make PSF velocity-invariant shift-invariant deconvolution
• Automatize PSF estimation
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 24
Flutter Shutter
[Raskar et al. 06]
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 25
Optimal Motion PSFs
• Optimality criteria PSF invertibility & estimation
[Agrawal & Xu 07]
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 26
Motion Invariant Photography
• Engineer PSF to be motion invariant
• Only for 1D motion
[Levin 08]
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 27
Hybrid Cameras
• Combined high-speed low-quality & low-speed high-quality camera
Input images Computed PSF
Deblurred Result Ground Truth
[Ben-Ezra & Nayar 04]
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 28
Motion Blur in Video
• Coded exposure &
super-resolution in
successive video frames
[Agrawal et al. 09]
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 29
Next: Further Light Properties
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 30
Flexible Voxels
• Flexible space-time resolution as post-processing
[Gupta et al. 10]
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 31
Synthetic Shutter Speed Imaging
• Combine multiple short exposures to reduce noise
• Align with optical flow
[Tel
leen
07]
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 32
Hybrid Cameras
• Motion deblurring & super-resolution
[Tai
et a
l. 08
]
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 33
Hybrid Cameras
• Motion deblurring & depth from two low-resolution high-speed camers
[Li e
t al.
08]
Input images
Deblurred result Recovered Depth
EG 2011 | Computational Plenoptic Imaging STAR | VI. High Speed Imaging 34
Analysis
• Analysis of optimal coded, single image deblurring
• MIP becomes worse when velocities exceed expectations
[Agrawal & Raskar 09]
Coded Exposure Motion Invariant Photography
Recommended