View
219
Download
0
Tags:
Embed Size (px)
Citation preview
Lapped TexturesLapped Textures
Emil PraunEmil Praun
Adam FinkelsteinAdam Finkelstein
Hugues HoppeHugues Hoppe
Emil PraunEmil Praun
Adam FinkelsteinAdam Finkelstein
Hugues HoppeHugues Hoppe
Princeton UniversityPrinceton University
Princeton UniversityPrinceton University
Microsoft ResearchMicrosoft Research
Princeton UniversityPrinceton University
Princeton UniversityPrinceton University
Microsoft ResearchMicrosoft Research
GoalGoal
“example” image“example” image
textured surfacetextured surface
mesh geometrymesh geometry
?
GoalGoal
Little user effortLittle user effort
No apparent seamsNo apparent seams
No obvious periodicityNo obvious periodicity
Low distortionLow distortion
Local texture controlLocal texture control
AnisotropyAnisotropy
Little user effortLittle user effort
No apparent seamsNo apparent seams
No obvious periodicityNo obvious periodicity
Low distortionLow distortion
Local texture controlLocal texture control
AnisotropyAnisotropy
Previous 2D Texture SynthesisPrevious 2D Texture Synthesis
Histogram equalization [Heeger ’96]Histogram equalization [Heeger ’96]
Laplacian block shuffling [de Bonet ’97]Laplacian block shuffling [de Bonet ’97]
Pixel template matching [Efros ’99] [Wei ’00]Pixel template matching [Efros ’99] [Wei ’00]
Histogram equalization [Heeger ’96]Histogram equalization [Heeger ’96]
Laplacian block shuffling [de Bonet ’97]Laplacian block shuffling [de Bonet ’97]
Pixel template matching [Efros ’99] [Wei ’00]Pixel template matching [Efros ’99] [Wei ’00]
Histogram equalization [Heeger ’96]Histogram equalization [Heeger ’96]
Laplacian block shuffling [de Bonet ’97]Laplacian block shuffling [de Bonet ’97]
Pixel template matching [Efros ’99] [Wei Pixel template matching [Efros ’99] [Wei ’00’00]]
Random pasting of image blocks [Xu ’00]Random pasting of image blocks [Xu ’00]
Histogram equalization [Heeger ’96]Histogram equalization [Heeger ’96]
Laplacian block shuffling [de Bonet ’97]Laplacian block shuffling [de Bonet ’97]
Pixel template matching [Efros ’99] [Wei Pixel template matching [Efros ’99] [Wei ’00’00]]
Random pasting of image blocks [Xu ’00]Random pasting of image blocks [Xu ’00]
Previous 2D Texture SynthesisPrevious 2D Texture Synthesis
Previous 3D TexturingPrevious 3D Texturing
Volumetric textures:Volumetric textures:
• Noise functions [Perlin ’85, Worley ’96]Noise functions [Perlin ’85, Worley ’96]
• Solid textures by example [Ghazanfarpour ’96]Solid textures by example [Ghazanfarpour ’96]
Synthesizing texture on a surface:Synthesizing texture on a surface:
• Reaction-diffusion [Turk ’91, Witkin ’91] Reaction-diffusion [Turk ’91, Witkin ’91]
• Cellular textures [Fleischer ’95]Cellular textures [Fleischer ’95]
• Covering surface with triangular tiles [Neyret ’99] Covering surface with triangular tiles [Neyret ’99]
Volumetric textures:Volumetric textures:
• Noise functions [Perlin ’85, Worley ’96]Noise functions [Perlin ’85, Worley ’96]
• Solid textures by example [Ghazanfarpour ’96]Solid textures by example [Ghazanfarpour ’96]
Synthesizing texture on a surface:Synthesizing texture on a surface:
• Reaction-diffusion [Turk ’91, Witkin ’91] Reaction-diffusion [Turk ’91, Witkin ’91]
• Cellular textures [Fleischer ’95]Cellular textures [Fleischer ’95]
• Covering surface with triangular tiles [Neyret ’99] Covering surface with triangular tiles [Neyret ’99]
ApproachApproach
texture patchtexture patch
surfacesurface
Key Idea: Patch PastingKey Idea: Patch Pasting
texture patchtexture patchtexture patchtexture patch
surfacesurfacesurfacesurface““lapped textures”lapped textures”
Anisotropic vs. Isotropic imagesAnisotropic vs. Isotropic images
PROCESSPROCESS
AlgorithmAlgorithm
Cut texture patches from input textureCut texture patches from input texture
[Specify texture direction & scale on mesh][Specify texture direction & scale on mesh]
REPEATREPEAT
Pick texture patch T, mesh point PPick texture patch T, mesh point P
Grow surface patch S around P to size(T)Grow surface patch S around P to size(T)
Flatten S over T; paste T on meshFlatten S over T; paste T on mesh
Update mesh coverage infoUpdate mesh coverage info
UNTIL mesh is coveredUNTIL mesh is covered
Cut texture patches from input textureCut texture patches from input texture
[Specify texture direction & scale on mesh][Specify texture direction & scale on mesh]
REPEATREPEAT
Pick texture patch T, mesh point PPick texture patch T, mesh point P
Grow surface patch S around P to size(T)Grow surface patch S around P to size(T)
Flatten S over T; paste T on meshFlatten S over T; paste T on mesh
Update mesh coverage infoUpdate mesh coverage info
UNTIL mesh is coveredUNTIL mesh is covered
AlgorithmAlgorithm
texture patchtexture patch
surfacesurface
AlgorithmAlgorithm
texture patchtexture patch
surfacesurface
AlgorithmAlgorithm
texture patchtexture patch
surfacesurface
AlgorithmAlgorithm
texture patchtexture patch
surfacesurface
IssuesIssues
1.1. Texture patch creationTexture patch creation
2.2. Specifying direction fieldSpecifying direction field
3.3. Surface patch growthSurface patch growth
4.4. Patch parametrizationPatch parametrization
5.5. Face coverage estimationFace coverage estimation
6.6. Texture storage and renderingTexture storage and rendering
1.1. Texture patch creationTexture patch creation
2.2. Specifying direction fieldSpecifying direction field
3.3. Surface patch growthSurface patch growth
4.4. Patch parametrizationPatch parametrization
5.5. Face coverage estimationFace coverage estimation
6.6. Texture storage and renderingTexture storage and rendering
IssuesIssues
1.1. Texture patch creationTexture patch creation
2.2. Specifying direction fieldSpecifying direction field
3.3. Surface patch growthSurface patch growth
4.4. Patch parametrizationPatch parametrization
5.5. Face coverage estimationFace coverage estimation
6.6. Texture storage and renderingTexture storage and rendering
1.1. Texture patch creationTexture patch creation
2.2. Specifying direction fieldSpecifying direction field
3.3. Surface patch growthSurface patch growth
4.4. Patch parametrizationPatch parametrization
5.5. Face coverage estimationFace coverage estimation
6.6. Texture storage and renderingTexture storage and rendering
Texture Patch CreationTexture Patch Creation
Less Structure SplotchLess Structure Splotch
IssuesIssues
1.1. Texture patch creationTexture patch creation
2.2. Specifying direction fieldSpecifying direction field
3.3. Surface patch growthSurface patch growth
4.4. Patch parametrizationPatch parametrization
5.5. Face coverage estimationFace coverage estimation
6.6. Texture storage and renderingTexture storage and rendering
1.1. Texture patch creationTexture patch creation
2.2. Specifying direction fieldSpecifying direction field
3.3. Surface patch growthSurface patch growth
4.4. Patch parametrizationPatch parametrization
5.5. Face coverage estimationFace coverage estimation
6.6. Texture storage and renderingTexture storage and rendering
Direction Field: User-specifiedDirection Field: User-specified
Direction Field: User-specifiedDirection Field: User-specified
• Angle - Only for anisotropic patches.Angle - Only for anisotropic patches.
• Can be set for only small number of Can be set for only small number of faces – Gaussian radial basis function faces – Gaussian radial basis function is used for interpolation.is used for interpolation.
• Since the normal lies within the plane of Since the normal lies within the plane of the face we get an orthogonal basis :the face we get an orthogonal basis :
S = T x N. (N – normal to the plane).S = T x N. (N – normal to the plane).
• Angle - Only for anisotropic patches.Angle - Only for anisotropic patches.
• Can be set for only small number of Can be set for only small number of faces – Gaussian radial basis function faces – Gaussian radial basis function is used for interpolation.is used for interpolation.
• Since the normal lies within the plane of Since the normal lies within the plane of the face we get an orthogonal basis :the face we get an orthogonal basis :
S = T x N. (N – normal to the plane).S = T x N. (N – normal to the plane).
Direction Field: Local to PatchDirection Field: Local to Patch
IssuesIssues
1.1. Texture patch creationTexture patch creation
2.2. Specifying direction fieldSpecifying direction field
3.3. Surface patch growthSurface patch growth
4.4. Patch parametrizationPatch parametrization
5.5. Face coverage estimationFace coverage estimation
6.6. Texture storage and renderingTexture storage and rendering
1.1. Texture patch creationTexture patch creation
2.2. Specifying direction fieldSpecifying direction field
3.3. Surface patch growthSurface patch growth
4.4. Patch parametrizationPatch parametrization
5.5. Face coverage estimationFace coverage estimation
6.6. Texture storage and renderingTexture storage and rendering
Patch GrowthPatch Growth
1. 1. Choose a random point on a non fully covered triangle.Choose a random point on a non fully covered triangle.
2. Add triangles in a BFS order to cover the texture patch.2. Add triangles in a BFS order to cover the texture patch.
3. Stop when the patch is fully coverd. 3. Stop when the patch is fully coverd.
Condintions:Condintions:
a.a. The surface patch homeomorphic to a disk.The surface patch homeomorphic to a disk.
b.b. The distortion should not exceed a given limit.The distortion should not exceed a given limit.
c.c. We add edge only if it is within the texture patch.We add edge only if it is within the texture patch.
1. 1. Choose a random point on a non fully covered triangle.Choose a random point on a non fully covered triangle.
2. Add triangles in a BFS order to cover the texture patch.2. Add triangles in a BFS order to cover the texture patch.
3. Stop when the patch is fully coverd. 3. Stop when the patch is fully coverd.
Condintions:Condintions:
a.a. The surface patch homeomorphic to a disk.The surface patch homeomorphic to a disk.
b.b. The distortion should not exceed a given limit.The distortion should not exceed a given limit.
c.c. We add edge only if it is within the texture patch.We add edge only if it is within the texture patch.
Patch GrowthPatch Growth
Patch GrowthPatch Growth
Patch GrowthPatch Growth
Patch GrowthPatch Growth
Patch GrowthPatch Growth
Patch GrowthPatch Growth
Patch GrowthPatch Growth
Patch GrowthPatch Growth
Polygonal HullsPolygonal Hulls
Start w/ boundary pixels, edges between Start w/ boundary pixels, edges between neighborsneighbors
Conservatively simplify polygonsConservatively simplify polygons
Start w/ boundary pixels, edges between Start w/ boundary pixels, edges between neighborsneighbors
Conservatively simplify polygonsConservatively simplify polygons
IssuesIssues
1.1. Texture patch creationTexture patch creation
2.2. Specifying direction fieldSpecifying direction field
3.3. Surface patch growthSurface patch growth
4.4. Patch parametrizationPatch parametrization
5.5. Face coverage estimationFace coverage estimation
6.6. Texture storage and renderingTexture storage and rendering
1.1. Texture patch creationTexture patch creation
2.2. Specifying direction fieldSpecifying direction field
3.3. Surface patch growthSurface patch growth
4.4. Patch parametrizationPatch parametrization
5.5. Face coverage estimationFace coverage estimation
6.6. Texture storage and renderingTexture storage and rendering
Align Patch to Direction FieldAlign Patch to Direction Field
texture patchtexture patch
surfacesurface
Tangential Vector FieldTangential Vector Field
Optimizing the ParametrizationOptimizing the Parametrization
Least squares best match to unit axesLeast squares best match to unit axesSparse linear system. No explicit fairness functionalSparse linear system. No explicit fairness functional
Least squares best match to unit axesLeast squares best match to unit axesSparse linear system. No explicit fairness functionalSparse linear system. No explicit fairness functional
Optimizing the ParametrizationOptimizing the Parametrization
TTTTSSSS
BBBB
CCCC
AAAA
((TT))((TT))
((SS))((SS))
((AA))((AA))
((BB))((BB))
((CC))((CC))
tttt
ssss
face in 3Dface in 3Dface in 3Dface in 3D 2D texture space2D texture space2D texture space2D texture space
Optimizing the ParametrizationOptimizing the Parametrization
For each mesh face For each mesh face f f ={A,B,C} the vector T ={A,B,C} the vector T lies within the face plain.lies within the face plain.
Barycentric coordinates:Barycentric coordinates:
T = T = A + A + B + B + C and C and + + + + = 0. = 0.
For each mesh face For each mesh face f f ={A,B,C} the vector T ={A,B,C} the vector T lies within the face plain.lies within the face plain.
Barycentric coordinates:Barycentric coordinates:
T = T = A + A + B + B + C and C and + + + + = 0. = 0.
Optimizing the ParametrizationOptimizing the Parametrization
TTTTSSSS
BBBB
CCCC
AAAA
((TT))((TT))
((SS))((SS))
((AA))((AA))
((BB))((BB))
((CC))((CC))
tttt
ssss
face in 3Dface in 3Dface in 3Dface in 3D 2D texture space2D texture space2D texture space2D texture space
ddTT = = (A) + (A) + (B) + (B) + (C) - (C) -
tt
^
We wish to minimize :We wish to minimize : f
ts dd22
Result of OptimizationResult of Optimization
IssuesIssues
1.1. Texture patch creationTexture patch creation
2.2. Specifying direction fieldSpecifying direction field
3.3. Surface patch growthSurface patch growth
4.4. Patch parametrizationPatch parametrization
5.5. Face coverage estimationFace coverage estimation
6.6. Texture storage and renderingTexture storage and rendering
1.1. Texture patch creationTexture patch creation
2.2. Specifying direction fieldSpecifying direction field
3.3. Surface patch growthSurface patch growth
4.4. Patch parametrizationPatch parametrization
5.5. Face coverage estimationFace coverage estimation
6.6. Texture storage and renderingTexture storage and rendering
Coverage estimationCoverage estimation
off-screen bufferoff-screen bufferoff-screen bufferoff-screen buffer
Render patch trianglesRender patch triangles
Flag covered trianglesFlag covered triangles
Remember 1 pixel Remember 1 pixel
per uncovered triangle per uncovered triangle
Render patch trianglesRender patch triangles
Flag covered trianglesFlag covered triangles
Remember 1 pixel Remember 1 pixel
per uncovered triangle per uncovered triangle
Coverage estimationCoverage estimation
off-screen bufferoff-screen bufferoff-screen bufferoff-screen buffer
Render patch trianglesRender patch triangles
Flag covered trianglesFlag covered triangles
Remember 1 pixel Remember 1 pixel
per uncovered triangle per uncovered triangle
Render patch trianglesRender patch triangles
Flag covered trianglesFlag covered triangles
Remember 1 pixel Remember 1 pixel
per uncovered triangle per uncovered triangle
Coverage estimationCoverage estimation
off-screen bufferoff-screen bufferoff-screen bufferoff-screen buffer
Render patch trianglesRender patch triangles
Flag covered trianglesFlag covered triangles
Remember 1 pixel Remember 1 pixel
per uncovered triangle per uncovered triangle
Render patch trianglesRender patch triangles
Flag covered trianglesFlag covered triangles
Remember 1 pixel Remember 1 pixel
per uncovered triangle per uncovered triangle
IssuesIssues
1.1. Texture patch creationTexture patch creation
2.2. Specifying direction fieldSpecifying direction field
3.3. Surface patch growthSurface patch growth
4.4. Patch parametrizationPatch parametrization
5.5. Face coverage estimationFace coverage estimation
6.6. Texture storage and renderingTexture storage and rendering
1.1. Texture patch creationTexture patch creation
2.2. Specifying direction fieldSpecifying direction field
3.3. Surface patch growthSurface patch growth
4.4. Patch parametrizationPatch parametrization
5.5. Face coverage estimationFace coverage estimation
6.6. Texture storage and renderingTexture storage and rendering
Texture Storage and RenderingTexture Storage and Rendering
Method 1: Method 1: Texture AtlasTexture Atlas
• Pre-composite into a global texture map.Pre-composite into a global texture map.
-- OR ---- OR --
Method 2: Method 2: Runtime pastingRuntime pasting
• Composite at run-time using hardwareComposite at run-time using hardware
Method 1: Method 1: Texture AtlasTexture Atlas
• Pre-composite into a global texture map.Pre-composite into a global texture map.
-- OR ---- OR --
Method 2: Method 2: Runtime pastingRuntime pasting
• Composite at run-time using hardwareComposite at run-time using hardware
Texture AtlasTexture Atlas
Texture Atlas is a set of charts {Texture Atlas is a set of charts {11,…, ,…, kk}.}.
Each chart Each chart j j is a mapping from a subset U is a mapping from a subset U
j j of the mesh to the Euclidean plane (the of the mesh to the Euclidean plane (the
texture plane).texture plane).
{U{U11,…, U,…, Ukk} is a cover of the surface.} is a cover of the surface.
If i If i j then j then j j and and i i don’t overlap except on don’t overlap except on
the edges.the edges.
Texture Atlas is a set of charts {Texture Atlas is a set of charts {11,…, ,…, kk}.}.
Each chart Each chart j j is a mapping from a subset U is a mapping from a subset U
j j of the mesh to the Euclidean plane (the of the mesh to the Euclidean plane (the
texture plane).texture plane).
{U{U11,…, U,…, Ukk} is a cover of the surface.} is a cover of the surface.
If i If i j then j then j j and and i i don’t overlap except on don’t overlap except on
the edges.the edges.
Texture AtlasTexture Atlas
Patches of triangles with similar normalsPatches of triangles with similar normals
2D packing problem for arbitrary polygons2D packing problem for arbitrary polygons
Patches of triangles with similar normalsPatches of triangles with similar normals
2D packing problem for arbitrary polygons2D packing problem for arbitrary polygons
Runtime PastingRuntime Pasting
Store vertex coordinates for each patchStore vertex coordinates for each patch
Composite at run-time using hardwareComposite at run-time using hardware
May render triangles several timesMay render triangles several times
Store vertex coordinates for each patchStore vertex coordinates for each patch
Composite at run-time using hardwareComposite at run-time using hardware
May render triangles several timesMay render triangles several times
Atlas vs. Runtime PastingAtlas vs. Runtime Pasting
AtlasAtlas
+ Faster rendering, more portableFaster rendering, more portable
+ Easy to paint unique details (eyes, nose on bunny)Easy to paint unique details (eyes, nose on bunny)
– Sampling artifacts; user effortSampling artifacts; user effort
PastingPasting
– Increases model complexity (Increases model complexity ( 1.6 –3)1.6 –3)
+ Huge effective resolution Huge effective resolution
+ Reuse splotch parameterization for many texturesReuse splotch parameterization for many textures
AtlasAtlas
+ Faster rendering, more portableFaster rendering, more portable
+ Easy to paint unique details (eyes, nose on bunny)Easy to paint unique details (eyes, nose on bunny)
– Sampling artifacts; user effortSampling artifacts; user effort
PastingPasting
– Increases model complexity (Increases model complexity ( 1.6 –3)1.6 –3)
+ Huge effective resolution Huge effective resolution
+ Reuse splotch parameterization for many texturesReuse splotch parameterization for many textures
RESULTSRESULTS
Results: SplotchesResults: Splotches
(completely automatic: no direction field)(completely automatic: no direction field)
Results: AnisotropicResults: Anisotropic
Controlling Direction and ScaleControlling Direction and Scale
Usage of more than 1 patchUsage of more than 1 patch
25 frames25 framesper sec!per sec!
tiny texturetiny texturefootprintfootprint
tiny texturetiny texturefootprintfootprint
LimitationsLimitations
low-frequencylow-frequencycomponentscomponents
boundaryboundarymismatchesmismatches
direction field direction field singularitiessingularities
TimingsTimings
Texture patch creation: Texture patch creation: 1 min1 min
Specifying direction field: Specifying direction field: 15 min15 min
Surface patch growthSurface patch growth
Patch parameterizationPatch parameterization
Face coverage estimationFace coverage estimation
Rendering: Rendering: 25fps25fps @ 1024 @ 102422
Texture patch creation: Texture patch creation: 1 min1 min
Specifying direction field: Specifying direction field: 15 min15 min
Surface patch growthSurface patch growth
Patch parameterizationPatch parameterization
Face coverage estimationFace coverage estimation
Rendering: Rendering: 25fps25fps @ 1024 @ 102422
Preprocessing:Preprocessing:
20sec – 6 min20sec – 6 min
Preprocessing:Preprocessing:
20sec – 6 min20sec – 6 min
Pentium III 733MHz, GeForce graphicsPentium III 733MHz, GeForce graphicsPentium III 733MHz, GeForce graphicsPentium III 733MHz, GeForce graphics
Human Human efforteffortHuman Human efforteffort
ConclusionsConclusions
Effective texture-by-example through:Effective texture-by-example through:
Overlapping texture patches Overlapping texture patches
Minimal edge blending Minimal edge blending
Aligning to direction field Aligning to direction field fast optimization fast optimization
Runtime compositing Runtime compositing high effective resolution high effective resolution
Effective texture-by-example through:Effective texture-by-example through:
Overlapping texture patches Overlapping texture patches
Minimal edge blending Minimal edge blending
Aligning to direction field Aligning to direction field fast optimization fast optimization
Runtime compositing Runtime compositing high effective resolution high effective resolution
Future WorkFuture Work
Other texture types:Other texture types:
• AnimatedAnimated
• ““Thick” (volumetric) textures Thick” (volumetric) textures fur fur
• NPR renderingNPR rendering
Fine-tuning patch placementFine-tuning patch placement
Greater automationGreater automation
Other texture types:Other texture types:
• AnimatedAnimated
• ““Thick” (volumetric) textures Thick” (volumetric) textures fur fur
• NPR renderingNPR rendering
Fine-tuning patch placementFine-tuning patch placement
Greater automationGreater automation