Texture Synthesis on [Arbitrary Manifold]

SurfacesPresented by:Presented by:

Sam Z. Glassenberg*Sam Z. Glassenberg*

* Several slides borrowed from Wei/Levoy presentation

Topics Investigated Thus Far

• 2D Texture Mapping (CS318)• 3D Textures (Hypertexture)• 2D Texture Synthesis

+

Synthesize a texture on a surface by coloring mesh vertices

Input Texture Input Mesh Result

Today’s Goal

Vertex Painting

• Advantages– Speed

• No additional transformation required

– Simplicity– Uniformly

distributed across geometry

• Disadvantages– Texture resolution

dependent on mesh complexity

2 SIGGRAPH Papers:

1. Texture Synthesis on Surfaces, by Greg Turk

2. Texture Synthesis over Arbitrary Manifold Surfaces, by Li-Yi Wei and Marc Levoy

Desirable Properties

• Share advantages of 2D algorithm– Quality– Efficient– General– Easy to use

• Minimum distortion• Minimum discontinuity

Differences betweenImages and Meshes

imageimage meshmesh

Synthesis Order

Pixels/Vertices

Local Orientation(Vector field)

?

?scanline

u

v

grid

Neighborhood

normal

? ?tangent bitangent

Wei/Levoy Solution

imageimage meshmesh

Synthesis Order

Pixels/Vertices

Local Orientation(Vector field)

?

?scanline

u

v

grid

Neighborhood

normal

? ?tangent bitangent

Wei and LevoyWei and Levoy

mesh re-tiling[Turk’92]

user-specifiedrelaxationrandom

random

flattening/resampling

Turk Solution

TurkTurk

repulsion[Turk’91]

User specified/interpolated

sweeping

Surface marching

imageimage meshmesh

Synthesis Order

Pixels/Vertices

Local Orientation(Vector field)

?

?scanline

u

v

grid

Neighborhood

normal

? ?tangent bitangent

Both Papers ExtendFast Texture Synthesis using Tree-structured Vector Quantization

Specifically by generalizing their definition of “search neighborhoods”

to apply to meshes

Texture Synthesis byNeighborhood Search

noise

Input pyramid

noise

Output pyramid

Search

Copy

Surface Texture Synthesis byNeighborhood Search

(Wei/Levoy)

Input pyramid Output pyramid

Search

Process

1. Build image/mesh pyramids

2. Assign texture orientation/Computation order

3. Generate texture

What aspects of image pyramids must we maintain in

mesh pyramids?

• Uniform density• Power-of-two complexity

differences between levels

Image & Mesh Pyramids

Mesh Retiling [Turk’92]

2 Turk-ish Methods for Mesh Retiling

• Turk ’92, used by Wei/Levoy– Uniformly

distributes mesh vertices

– Requires “shooting normals” to move between levels

• Turk ’91, used by Turk– Uniformly

distributes mesh vertices

– Maintains parent/child relationship between levels

Retiling Density

24576 vertices 73728 vertices

Turk ‘91• Create a mesh hierarchy in which mesh

Mk = (Vk, Tk) is defined by its Vertices and Triangles– For the lowest mesh in the hierarchy, place n

points on the surface– Use repulsion to distribute points easily– Add 3n points to make the next level– Repeat

• Connect points by projecting nearby points onto a tangent plane

• Perform Delaunay triangulation to reduce triangles

Now, we need to determine orientation…

In Turk’s method, we use this orientation to determine the computation order (Surface

Sweeping)

In Wei/Levoy’s method, orientation is needed to “flatten” the neighborhoods.

Texture Orientation

• Generate a coordinate frame• Three orthogonal axes

– s (texture right)– t (texture up)– n (surface normal)

Texture Orientation

• Methods for orienting textures – user-specified (Turk)– random (Wei/Levoy)– smooth or symmetric (Wei/Levoy)

Texture Orientation (User-specified)

4-way symmetric texture 4-way symmetric vector field

Texture Orientation (Symmetry)

Texture Orientation

random 2-way symmetry4-way symmetry

Texture Orientation (Relaxation)

• Minimize an error function

Results (Wei/Levoy):Random Orientation

Results (Wei/Levoy):Other Orientations

Random User-specified Relaxation

2-way symmetry

4-way symmetry

Texture Synthesis on Surfaces:Wei/Levoy Style

Synthesis

Synthesis : 2 Lowest Levels

Synthesis : Lowest Level

Random copy

Synthesis Pass1 : Extrapolation

Search

Copy

Shooting normal

Mesh Neighborhood

Resample

Compare

Resampled Grid

2D Patch

3D Patch

Flatten (Maillot’93)

Neighborhood Flattening

• Project the triangles adjacent to p onto p’s local texture coordinate system

• Add triangles one-at-a-time until neighborhood template is covered

Neighborhood Comparison

compare ?

Synthesis Pass 2 :Full Neighborhood

Search

Copy

Multiresolution Synthesis

Texture Synthesis on Surfaces:According to Turk

Turk’s Surface Sweeping

• Select an anchor vertex A• Assign s(v) = sweep distance to vertex v along

the vector field from A (for all v)• Consensus orientation b/w 2 vertices v and w:

Ovw = (O(v) + O(w))/2

• Calculate a new s(v) as a weighted average that its neighboring values dictate it should have.

• Visit vertices in order by sweep distance

Turk’s Texture Synthesis (Pseudocode)

I = Input textureN(v) = Neighborhood around vM(a,b) = Neighborhood around (a,b)D(M,N) = Match Value (sum of squared differences)

Turk’s Mesh Neighborhoods

• r = average distance between mesh vertices• O(v) = Surface tangent vector• P(v) = O(v) rotated 90o about surface normal• Together, O and P make a coordinate frame !

u

vnormal

P O• Now, we can traverse the surface by

point repelling

Point Repelling

• Use color interpolation to determine color at current surface point

• Move r in the direction of O or P• When an edge is reached, fold the

path over the next polygon

Results

smooth 4-way symmetry random

Results

Surface displacement

smooth 2-way symmetry 2-way symmetry

Results

Summary of Differences

Turk’s approach Our approach

Vector field

Traversal order

Neighborhood

Mesh hierarchy

smooth random, symmetric

sweeping random

surface marchingflattening/resampling

explicit parent/child shooting normal