Hundreds of thousands of polygons! Hundreds of thousands of polygons! Painstaking detail for the...

Byong Mok OhByong Mok Oh Max Chen Max Chen Julie DorseyJulie Dorsey Frédo DurandFrédo Durand

• Hundreds of thousands of polygons!Hundreds of thousands of polygons!• Painstaking detail for the artist to create.Painstaking detail for the artist to create.• Something’s not quite convincing.Something’s not quite convincing.

• Already photorealistic.Already photorealistic.• Detail exists before the artist begins to Detail exists before the artist begins to work.work.

Just follow these simple steps!Just follow these simple steps!• Separate the source image into layers.Separate the source image into layers.• Assign depth.Assign depth.• Adjust lighting.Adjust lighting.

Using two proprietary tools, Using two proprietary tools, The Clone The Clone Brushing ToolBrushing Tool & & The Texture-Illuminance The Texture-Illuminance Decoupling FilterDecoupling Filter, a single image can be , a single image can be transformed into a three dimensional transformed into a three dimensional scene.scene.

Layering is over 75% Layering is over 75% of the work!of the work!

A reference location is set, A reference location is set, and each layer of depth is and each layer of depth is used to create perspective.used to create perspective.

• Artists can manually adjust Artists can manually adjust depth by, “chiseling”. depth by, “chiseling”. • Depth paint is blended for Depth paint is blended for smooth transitions.smooth transitions.

• To ensure accuracy, a planar surface is To ensure accuracy, a planar surface is used as a reference point.used as a reference point.• A map is generated to represent the A map is generated to represent the surface depth.surface depth.• All objects’ depth will be based on this All objects’ depth will be based on this surface.surface.

To create depth in simple objects, To create depth in simple objects, spheres, cylinders and cubes are spheres, cylinders and cubes are used mask depth.used mask depth.

For more organic looking objects, For more organic looking objects, level sets are used.level sets are used.

Level sets break an object into Level sets break an object into levels from the surface inwards, levels from the surface inwards, allowing the object to “bulge” out allowing the object to “bulge” out around the edges, creating fuller, around the edges, creating fuller, more life-like depth.more life-like depth.

Artists can tweak the depth settings and Artists can tweak the depth settings and paint additional depth, or remove added paint additional depth, or remove added depth to both organic and primitive depth to both organic and primitive geometry.geometry.

Much like Adobe Photoshop’s Much like Adobe Photoshop’s “Rubber Stamp” tool, Clone “Rubber Stamp” tool, Clone Brushing allows an artist to paint Brushing allows an artist to paint one part of an image with the one part of an image with the color of another.color of another.

This does not handle lighting or This does not handle lighting or depth changes very well.depth changes very well.

Using color primarily from areas of Using color primarily from areas of similar depth and orientation, similar depth and orientation, distortion is minimized.distortion is minimized.

A reference image or layer is used A reference image or layer is used as a texture. as a texture.

Brushing becomes real time texture Brushing becomes real time texture mapping. mapping.

U & V coordinates from the source U & V coordinates from the source are painted onto 3D objects.are painted onto 3D objects.

4 pixels are sampled for each 4 pixels are sampled for each pixel painted, allowing the pixel painted, allowing the system to interpolate color system to interpolate color with some accuracy. with some accuracy.

Only bilinear interpolation is Only bilinear interpolation is implemented, so textures can implemented, so textures can be painted in real time.be painted in real time.

Pixels being painted are treated as Pixels being painted are treated as “active” pixels. “active” pixels.

Any painting over these pixels is Any painting over these pixels is smoothed linearly, giving a natural look, smoothed linearly, giving a natural look, without high overhead.without high overhead.

One pixel ahead of each active pixel is One pixel ahead of each active pixel is sampled prior to being painted. sampled prior to being painted.

This allows the system to predict what will This allows the system to predict what will be painted next, in any direction, speeding be painted next, in any direction, speeding up performance.up performance.

Painted pixels Painted pixels cannot be cannot be sampled. sampled. Textured objects Textured objects are “frozen”.are “frozen”.

• The system can identify the major The system can identify the major light sources in a scene.light sources in a scene.

• Light can then be added, removed, Light can then be added, removed, replicated or altered.replicated or altered.

Once the major light sources in a Once the major light sources in a scene are identified, their effects scene are identified, their effects can be negated, and individual can be negated, and individual objects can be discerned from a objects can be discerned from a scene.scene.

Understanding the light in a Understanding the light in a scene greatly enhances the scene greatly enhances the accuracy of the scene’s depth.accuracy of the scene’s depth.

Soft shadows aren’t always Soft shadows aren’t always distinguishable. Filtering distinguishable. Filtering helps smooth the transition helps smooth the transition between a soft shadow between a soft shadow and an object.and an object.

Even with filtering, some Even with filtering, some shadows will still be visible.shadows will still be visible.

Originally implemented as an Originally implemented as an Apple Quick Time Virtual Tour, Apple Quick Time Virtual Tour, panoramic stitching was the panoramic stitching was the first implementation of Image first implementation of Image Based Modeling. Based Modeling.

With this system, each object in With this system, each object in a scene like this could be a scene like this could be modeled.modeled.

Stereo images have Stereo images have been used, with been used, with limited success, to limited success, to generate 3D models.generate 3D models.

With Non-Distorted Clone With Non-Distorted Clone Brushing & Texture-Illuminance Brushing & Texture-Illuminance Decoupling Filters, a polygon Decoupling Filters, a polygon efficient 3D scene can be efficient 3D scene can be generated from an image like this generated from an image like this within 13 hours.within 13 hours.