Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
1
La photographienumérique
Frank NIELSENLundi 7 Juin 2010
Le Monde digitalLe Monde digitalKey benefits of the analog2digital paradigm shift?
• Dissociate contents from support : “binarize”• Universal playerUniversal player (CPU, Turing machine)• Generic algorithms
Le Monde numériqueMonde numérique omnipresent, informatique ubiquitaire
Nouveautés
Numérique = Digital + Calcul
Exemple:Image numérique
(=calculée)
Photographie numérique : Une profonde Photographie numérique : Une profonde révolution a venir
??Photography Computational Photography
Pas seulement dans le domaine grand public… … mais aussi dans beaucoup d’autres domaines des sciences
Computational anatomy(differential geometry)
Example 1Non Photo Realistic Camera
Introduction: What’s computational photography?Introduction: What’s computational photography?
??
Warm up: Warm up: Nonphotorealistic camera (NPR camera) Nonphotorealistic camera (NPR camera) Multiple flashes to easily get depth discontinuities
Baseline 50mm (depth 5mm at 2 meters)
Canny edge detector
(for comparison only)
Multiflash depth edgeStylized rendering
Source image
Shadow to the leftShadow to the right
Remove shadows, show lesionfrom depth discontinuity analysis
Warm up: Nonphotorealistic (NPR) cameraWarm up: Nonphotorealistic (NPR) cameraLaparoscope camera with two fiber optics lightingtwo fiber optics lighting
Difficult to analyze using traditional image processing techniques
A Non-Photorealistic Camera: Depth Edge Detection and Stylized Rendering with Multi-Flash Imaging.SIGGRAPH 2004 http://www.merl.com/people/raskar/NprCamera/
Example 2Synthetic Aperture Focusing Camera
Introduction: What’s computational Introduction: What’s computational photography?photography?
??
Camera array provides many individual apertures → synthetic aperture focusing
Aperture means beyond pinhole camera algorithms
Warm up: Synthetic aperture focusing camera Warm up: Synthetic aperture focusing camera (SAF)(SAF)
High Performance Imaging Using Large Camera Arrays. SIGGRAPH 2005http://graphics.stanford.edu/projects/array/
focal length
sensorsize ap
ertu
reap
ertu
re
lens( SAF camera: 1-shot many images!)
Warm up: Synthetic aperture focusing Warm up: Synthetic aperture focusing cameracamera(SAF)(SAF)
Synthetic aperture focusing
+Averaging multiple images also improve Signal-to-Noise ratio (SNR)
SAF is good enough for image recognition
Σ
Single camera aperture
Synthetic (=calcul) aperture
3D World Sensor planeCamera array
Example 3Shape-Time Camera
(Depict the world)
Introduction: What’s computational photography?Introduction: What’s computational photography?
??
Depict world in new ways: Shape-time photography
(burst-mode on stereo adaptor)
Warm up: Depicting the worldWarm up: Depicting the world
Shape-time photography. CVPR 2003people.csail.mit.edu/billf/
Picasso Hockney
Stereo mount Depiction
Matte extraction: strobing application
Old film of Etienne-Jules Marey
Visualizing motion is important for video-based applications (PVR,etc.)
Mosaicing+matting provides a kinetic experience
Comp. Photography: Novel World DepictionsComp. Photography: Novel World Depictions
Computer generated motion lines
Comp. Photography: Novel World DepictionsComp. Photography: Novel World Depictions
Computational Photography: Motion amplificationComputational Photography: Motion amplification
Motion magnification, SIGGRAPH 2005http://people.csail.mit.edu/celiu/motionmag/motionmag.html
A video example best described the result(Applications to telesurveillance, etc.)
Computational Photography: Motion amplificationComputational Photography: Motion amplification
Motion magnification, SIGGRAPH 2005http://people.csail.mit.edu/celiu/motionmag/motionmag.html
Inpainting… Texture Synthesis…
Hallucination….
Computational PhotographyComputational Photography
Region filling and object removal by exemplar-based inpainting. IEEE Trans. Image Process. 2004http://research.microsoft.com/~antcrim/papers.htm
Computational Photography: Computational Photography: ClickRemovalClickRemoval applet applet
Frank Nielsen, Richard Nock: ClickRemoval: interactive pinpoint image object removal. ACM Multimedia 2005:
Demo
Image retargettingImage retargettingAdjust contents to screen size(TV, PDA, Phone, etc.)
SIGGRAPH 2007
Demo
Human Perceptionversus
Digital Image Processing
Computational Photography: Human PerceptionComputational Photography: Human Perception
S/W Computational Photo.: Hybrid imagesS/W Computational Photo.: Hybrid images
Hybrid images, SIGGRAPH 2006.
Low frequencyat far distance
High frequencyat close distance
Overriding Dynamic range… Tone mappingOverriding Dynamic range… Tone mappingScientific (measurement) images ≠ Human perceptual images
Image segmentation and lightness perception, Nature 434, 79-83, 2005
‘light disks’ visible through dark haze
‘dark disks’ visible through light haze
Disks are exactly identicalbut are perceived differently
Computational PhotographyNovel hardware & processing techniques
Computational Photography: H/WComputational Photography: H/W
Coaxial Infrared camera + ProjectorTranscoding (pseudo-coloring)Coaxial Infrared camera + ProjectorTranscoding (pseudo-coloring)
VeinViewer (Luminetx) | http://www.luminetx.com/main/default.aspx
Computational Photography: Vein ViewerComputational Photography: Vein Viewer
Computational Photography:Computational Photography:
Computing inOptical Domain
H/W Comp. Photo.: Computing in Optical domainH/W Comp. Photo.: Computing in Optical domain
Control the rays in space-time:Exposure allows optical computations→Light integration on the sensor
Programmable imaging using a digital micromirror array (CVPR’04)Programmable Imaging: Towards a Flexible Camera, Int. Journal of Computer Vision. 2006
H/W Comp. Photo.: Computing in Optical domainH/W Comp. Photo.: Computing in Optical domain
Programmable imaging using a digital micromirror array (CVPR’04)Programmable Imaging: Towards a Flexible Camera, Int. Journal of Computer Vision. 2006
Require to calibrate the DMDwith the camera coarsely
Convolution in optical domain Convolution in optical domainfor face recognition
Computational Photography:Computational Photography:
Computing withexotic “lenses”
Control the light rays on each layer:
Multiple-layer aperture
Computational Photo.: Lensless CameraComputational Photo.: Lensless Camera
Lensless Imaging with a Controllable Aperture, CVPR 2006
Traditional New
Computational Photo.: Lensless CameraComputational Photo.: Lensless Camera
Lensless Imaging with a Controllable Aperture, CVPR 2006
Pan/tilt field of view (fov) without physical moving parts
Split field of view, spatially varying zoom
Computations in optical domain
Computational Photo.: Lensless CameraComputational Photo.: Lensless Camera
Lensless Imaging with a Controllable Aperture, CVPR 2006
Appearances of eyes captures both the environment and gazing direction
Spherical panorama(latitude-longitude)
Corneal Imaging System Environment from Eyes, Int. Journal on Computer Vision (IJCV) 2006.Eyes for relighting, SIGGRAPH 2004.
Computational Photography: Eye OpticsComputational Photography: Eye Optics
Comp. Photography: Radial Catadioptric CameraComp. Photography: Radial Catadioptric Camera
Multiview Radial Catadioptric Imaging for Scene Capture SIGGRAPH 2006
• Capture a radial space of rays• Both mirrored and object parts• 3D reconstruction with BRDF(using a single shot!)
Computational Photography:Computational Photography:
Beyond 2D pixels:4D+ Light fields
The Plenoptic Function and the Elements of Early Vision 1991Plenoptic Modeling: An Image-Based Rendering System, SIGGRAPH 1995
Computational Photography: Plenoptic cameraComputational Photography: Plenoptic camera
Plenoptic (latin plenus+optics) is a 7D function (X,Y,Z,θ,φ,λ,t)
16 MP: 300x300 lens images
Moving the viewpoint
Digital refocusing
Acquire first, postprocess later.
Computational Photography: Light field cameraComputational Photography: Light field camera
Fourier Slice Photography, SIGGRAPH 2006
H/WComp. Photography: Light field cameraH/WComp. Photography: Light field camera
Fourier Slice Photography, SIGGRAPH 2006
Fourier Slice Photography
Lens Sensor Image
Generalizedoptics
Computationalsensor
Computational imaging
Display
Novel displays
Computational Photography: Computational Photography: Images in the 21Images in the 21stst Century Century
Image numérique
= calcul