View
218
Download
0
Category
Preview:
Citation preview
1
Applications
Shmuel Peleg and Joshua Herman, “Panoramic Mosaics by Manifold Projection”, Computer Vision and Pattern Recognition (CVPR), 1997
Wolfgang Heidrich and Hans-Peter Seidel, “View-independent environment maps”, SIGGRAPH / Eurographics Workshop on Graphics Hardware, 1998
Matthew Brand, “Charting a Manifold”, Mitsubishi tech report, 2003
K.Grochow, S. Martin, A. Hertzmann, and Z. PopovicStyle-based Inverse Kinematics, Siggraph 2004
2 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Applications using manifolds
• Many problems can be phrased in manifold terminology• Provides an alternative way of viewing the
problem• Can also provide some formalism
3 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Applications
•Surface fitting• Consistent parameterization
•Image-based rendering•Environment mapping•Animation
4 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Surface fitting•CT, MRI data, noisy, densely sampled in widely-spaced contours
• Boundary conditions less problematic
Fitting Manifold Surfaces To 3D Point CloudsCindy Grimm, David Laidlaw, and Joseph CriscoJ. of Biomedical Engineering Feb. 2002
5 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Consistent parameterization
•Build manifold once• Fit to multiple bone point data sets
6 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Aligning parameterization
•Identify similar points on meshes• Pin parameterization
7 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Application: Panoramas
•Problem statement:• Given images from a known camera movement
• Rotation about camera axis• “Push-broom” pan (assumes negligible depth)
• “Glue” images together into a single image
Rover, nasa.gov
8 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Camera rotation
•Final image can be rendered on a cylinder• No parallax• Each image samples some number of pixels on
cylinder (manifold) image
Peleg and Herman
9 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Push broom/vertical slit camera•Translation of camera
• Image slit perpendicular to camera motion• Need not travel in straight line
• Depth differences negligible• Parallax
• Manifold is part of ground plane viewed by camera
Ground plane
Direction of travel
Peleg and Herman
10 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Practical problem
•How to line up individual images to create one seamless image?
• Manifold: Final image (3D function RGB on 2D manifold)
• Charts: Individual images (2D charts)• Overlap regions/transition functions: Unknown
• Assume translation• (Account for optical effects of camera)
•Note: Only works for these two camera motions
11 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
General solution
•Define a format for the transition function• E.g., translation in x,y
•Define an error metric that measures how well two overlap regions agree
• E.g., pixel difference•Optimize over free parameters in transition function
• E.g., x,y shift between all pairs of overlapping images
12 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Solving for overlaps, transitions
•Find translation that minimizes pixel differences
• Find that minimizes || I0(s) – I1((s))||
• (s) = s + s, where s is unknown
0 1
0 1
s
13 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Final image
•Transition functions align images (abstract manifold)• Final image colors? (RGB function on manifold)
•Blend and embedding functions for each chart• Embedding function: Original image• Blending function: How much to use of each
overlapping image• Usually favor very short blend regions
15 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Application: Environment mapping
•Place scene/model inside sphere•Light intensity/color found by intersecting normal with sphere
• 1-1 mapping between normal direction and sphere• Every point on sphere assigned light intensity/color
•Implementation• Store colors in one (or more) texture maps (2D)
16 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Parameterization
•Surface normal (point on sphere) to point in texture map• Atlas/local parameterization
•Desirable properties• Even sampling of sphere
• Adaptive• Partition
• Overlap (mip mapping, continuity)• Simple to compute
• Amenable to GPU implementation
17 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Approach 1
•Single texture map• Not unique (poles)• Poor sampling• Simple to compute
18 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Approach II
•Cube mapping• Six charts• Discontinuities at edges• Sampling better at center of faces than edges• Simple (plane) computation
• Which plane?
19 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Approach III
•Parabolic mapping• Chart functions use parabolic
function• Better sampling• Slightly more computation• Less-noticeable seams
Heidrich and Hans-Peter Seidel
20 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Proposal
•Use chart approach• Allows for adaptive sampling (more detail where
needed)• Chart sizes uniform: Tile texture map• Include overlap
• Minimal extra texture map• Mip-mapping/down sampling
21 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Application: Animation
•Human configuration space lies on a manifold of dimension n embedded in m dimensional space, where n << m•Articulated skeleton: over 40 degrees of freedom (shoulders, knees, hips, etc., each 1-3 degrees of rotation)•Individual motions (reaching, walking) certainly lie on lower dimension manifolds
• End-point of reach plus time•Shape of manifold of all possible human motions?
• Who knows? K.Grochow, S. Martin, A. Hertzmann, and Z. Popovic
22 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Overview
•Manifold learning• Data samples (e.g., motion capture, key frames)
• Interpolation equals manifold construction• Editing equals manifold editing
• High-level review of manifold learning techniques• What kind of manifolds can you learn/construct
from data?•2D animation example•Manifolds in animation
23 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
2D illustration
•Two joint angles• Circle X Circle manifold (torus)
•Animation• Repetitive motion• Joint angle plot
• Circle manifold•Animation is a 1D manifold embedded in 2D
24 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
2D illustration
•Two joint angles• Circle X Circle manifold (torus)
•Animation• Repetitive motion• Joint angle plot
• Circle manifold•Animation is a 1D manifold embedded in 2D
25 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Manifold learning
•Input: Sample points in Rm
• E.g., Motion capture sequence, each pose is a data point, m is number of dof of joints• 2D example: , for each pose
•Assume data lies on a manifold of dimension n• Constraints on manifold shape/geometry (e.g.,
linear, no self-intersections)•Goal: Parameterize manifold
• Single chart
26 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Manifold learning, linear techniques
•Principal components analysis (PCA), Independent components analysis (ICA)
• Assumes shape of manifold in space is linear/planar
• Learn vector(s) that span the hyper-plane
27 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Manifold learning, deformed linear
•Support vector machines (SVM)• Deform space points are in first
• Deformation is “nice”• Warps points to new places• “Guess” which deformation will work
• Learn linear/planar function
Deformation Linear parameterization
28 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Manifold learning, local shape
•Isomap, Local linear embedding (LLE), Semi-definite embedding (SDE)
• Define a distance metric between points• Assumptions:
• K closest neighbors are also closest K neighbors on the manifold
• Disk around point in manifold would contain all K neighbors, but not others
• Know dimension n of manifold• Possible to deform K neighbors into n dimensions while maintaining
(relative) distances
•Output: Single chart/parameterization of entire data set
29 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Isomap, LLE, SDE cont.
•Non-obvious failure modes• Circular/repetitive data sets• Self-intersections
Raw result: 2D embedding
Modified : 1D embedding
30 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Manifold construction as learning
•Use K neighbors to define chart domains (Uc)
• Charts are “squished” Gaussians• Center, tangent vectors
•Find transition functions (affine transformations)• Transformation takes tangent vectors into Rn
• Aligns free vectors with neighbors
UcMatthew Brand, “Charting a Manifold”, Mitsubishi tech report, 2003
31 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Uses of animation manifold•General idea:
• Construct (implicitly or explicitly) a manifold representing valid human poses
• Create a new animation sequence• Foot must touch here, reach here, etc.
• Not sufficient to constrain all degrees of freedom (dof)
• Project on to manifold to fill in remaining dof
K. Grochow, S. L. Martin, A. Hertzmann, Z. Popovic,Style-Based Inverse Kinematics, Siggraph 2004
32 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Re-sequencing as embedded manifold
•Goal: Given existing sequence (samples), add more/change samples•Assumptions:
• Samples come from some smooth manifold• Some form of interpolation gives new samples on
manifold•Current approaches: Interpolation between neighboring samples in sequence for given new time
33 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Re-phrasing problem
•Manifold learning or sequence timing provides parameterization/abstract manifold•Embed manifold with smooth function
• Parameterization• Use function fitting
•Re-sequencing: Evaluate embedding function
E(M)
34 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Caveats
•What makes animation data difficult?• “Distance” loses meaning in >> 10 dimensions
• Every point equally far away• Can’t enumerate
• Noise• Error in capture process• Skeleton only approximates human motion
• Joint angle representation• Don’t explicitly deal with manifold,
parameterization
35 Siggraph 2005, 8/1/2005www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html
Summary
•Manifolds provide a formalism for breaking a problem into manageable pieces
• Charts provide local parameterization• Planar domains
• Overlaps: Natural mechanism for moving between parameterizations• Blend functions instead of geometric constraints• No boundary condition problems
•Explicitly encapsulating/representing manifold is beneficial
• Cleaner algorithm specifications
Recommended