Embed Size (px)
Citation preview
Dr Yann Savoye
Innsbruck University
POEMS´15: Polytopal Element Methods inMathematics and Engineering
October 30, 2015 www.animlife.com
"Free at Last"
Cage-based Living Geometry
October 30, 2015 #2
Content
1. Introduction2. Background and Related Work3. Cage-based Inversion4. Cage-based Reskinning5. Cage-based Registration6. Cage-based Stylization7. Conclusions
Dr Yann Savoye
1
October 30, 2015 Dr Yann Savoye #3
Bringing Life to Creatures
• Capturing Life-Like Dynamic Geometry Into Deformable Cages.• A new Graphics-To-Vision Strategy built around the cage-based scheme.• Several Technical Tools in Shape Modeling, Animation, 4D Vision.
October 30, 2015 Dr Yann Savoye #4
Bringing Life to Creatures
October 30, 2015 Dr Yann Savoye #5
Technical Pillars
October 30, 2015 Dr Yann Savoye #6
Inspired from Traditional Animation
October 30, 2015 #7
Content
1. Introduction2. Background and Related Work3. Cage-based Inversion4. Cage-based Reskinning5. Cage-based Registration6. Cage-based Stylization7. Conclusions
Dr Yann Savoye
2
October 30, 2015 #8
Cage-based Performance Capture.
Dr Yann Savoye
October 30, 2015 Dr Yann Savoye #9
Cage-based Deformation
Mean Value CoordinatesJu et al ‘05
Green CoordinatesLipman et al ‘08
Variational HarmonicsBen Chen et al ‘09 Harmonic Coordinates
Joshi et al ‘07
Biharmonic CoordinatesJacobson et al ‘07
October 30, 2015 Dr Yann Savoye #10
Mean Value Coordinates
Mean Value Coordinates for Closed Triangular Meshes
Ju et al ‘05
October 30, 2015 Dr Yann Savoye #11
Green Coordinates
Green CoordinatesLipman et al ‘08
October 30, 2015 Dr Yann Savoye #12
Harmonic Coordinates
Harmonic Coordinates for Character ArticulationJoshi et al ‘07
October 30, 2015 #13
Presentation Overview
Dr Yann Savoye
Cage-basedModeling
Cage-based Reskinning
Cage-basedRegistration
Cage-basedStylization
Interactive Modeling(User-Specified)
Animation Reskinning(Temporally Data-Driven)
Space-Time Capture(Video-based, Unsupervized)
Surface Exaggeration( Space-Time Filtering )
October 30, 2015 #14
Presentation Overview
Dr Yann Savoye
Cage-basedModeling
Cage-based Reskinning
Cage-basedRegistration
Cage-basedStylization
Graphics Vision
October 30, 2015 #15
Presentation Overview
Dr Yann Savoye
Cage-basedModeling
Cage-based Reskinning
Cage-basedRegistration
Cage-basedStylization
Graphics-To-Vision Strategy: Relaxing the prior knowledge on the data
October 30, 2015 #16
Content
1. Introduction2. Background and Related Work3. Cage-based Inversion4. Cage-based Reskinning5. Cage-based Registration6. Cage-based Stylization7. Conclusions
Dr Yann Savoye
3
October 30, 2015 #17
Cage-based Modeling
Dr Yann Savoye
Cage-based Reskinning
Cage-basedRegistration
Cage-basedStylization
Objective: Introduction of a core inverse cage-based technique.
Cage-basedModeling
October 30, 2015 Dr Yann Savoye #18
Interactive Shape Modeling
Laplacian Mesh EditingSorkine et al ‘05
Volumetric Graph LaplacianZhou et al ‘05
As Rigid As PossibleSorkine et al ‘07
Mesh IKSumner et al ‘05
October 30, 2015 Dr Yann Savoye #19
Inverse Cage-based Modeling
Sparse Contraints
Default Pose Resulting Deformation
October 30, 2015 Dr Yann Savoye #20
Inverse Cage-based Modeling
• Least-Square Cage (regularisation term)
October 30, 2015 Dr Yann Savoye #21
Inverse Cage-based Modeling
• Differential Cage Coordinates
• Laplacian Cage Operator
October 30, 2015 Dr Yann Savoye #22
Inverse Cage-based Modeling
• User-Specified Deforming Constraints
October 30, 2015 Dr Yann Savoye #23
GPU Barycentric Map
October 30, 2015 Dr Yann Savoye #24
Inverse Cage-based Modeling
• Cage-based Fitting
October 30, 2015 Dr Yann Savoye #25
Inverse Cage-based Modeling
• Sparse Linear Least-Square Fitting
October 30, 2015 #26
Content
1. Introduction2. Background and Related Work3. Cage-based Inversion4. Cage-based Reskinning5. Cage-based Registration6. Cage-based Stylization7. Conclusions
Dr Yann Savoye
4
October 30, 2015 #27
Cage-based Reskinning
Dr Yann Savoye
Cage-basedRegistration
Cage-basedStylization
Cage-basedModeling
Objective: Introduction of a core space-time cage fitting technique.
Cage-based Reskinning
October 30, 2015 Dr Yann Savoye #28
Surface Performance Capture
Surface CaptureStarck and Hilton ‘07 Skin Capture
Park and Hodgins ‘06
Performance CaptureDe Aguiar et al. “08
October 30, 2015 Dr Yann Savoye #29
Highly Non-Rigid Surfaces
October 30, 2015 Dr Yann Savoye #30
Generic and Shape-Aware Cages
October 30, 2015 Dr Yann Savoye #31
Harmonic Coordinates
Harmonic Rigging
Cage-Model Setup Cage Voxelization Volumetric Heat diffusion
October 30, 2015 Dr Yann Savoye #32
Cage-based Performance Animation
• Space-Time Cage Optimization
October 30, 2015 Dr Yann Savoye #33
Silhouette Consistency
• Evaluation: Silhouette Consistency
October 30, 2015 #34
Content
1. Introduction2. Background and Related Work3. Cage-based Inversion4. Cage-based Reskinning5. Cage-based Registration6. Cage-based Stylization7. Conclusions
Dr Yann Savoye
5
October 30, 2015 #35
Cage-based Registration
Dr Yann Savoye
Cage-basedStylization
Cage-basedModeling
Cage-based Reskinning
Objective: Extending the system toward unaligned geometric targets.
Cage-basedRegistration
October 30, 2015 Dr Yann Savoye #36
Geometric Reconstruction
Polyhedral Visual HullMatusik et al ‘01
Watertight ReconstructionHornung et al ‘07
Multi-view StereoLiu et al ‘10
October 30, 2015 Dr Yann Savoye #37
Geometric Registration
Piecewise Rigid registrationPekelny et al ‘08
Non-Rigid RegistrationLi et al ‘08
Patch-based RegistrationBudd et al ‘10
October 30, 2015 Dr Yann Savoye #38
• Camera Projection Operator
• Silhouette-based Visual Cone
• Voxel-based Visual Hull
Target Geometry from Silhouettes
October 30, 2015 Dr Yann Savoye #39
Volumetric Shape Carving
October 30, 2015 Dr Yann Savoye #40
Watertight Surface Polygonization
• Temporally Inconsistent Surface
• Laplacian Surface Smoothing
• Oriented Target Point Clouds
October 30, 2015 Dr Yann Savoye #41
Target Geometry from Silhouettes
October 30, 2015 Dr Yann Savoye #42
Normal-Guided Correspondences
• Normal-Guided Pruning.• Standard Shepard Interpolation for compatible candidates. • Locally Weighted Correspondences.
October 30, 2015 Dr Yann Savoye #43
• Biharmonic Cage-based Coordinates
• Per-Handle Biharmonic Field
• Flexible Local Rigidity Field
Biharmonic Overlapping Fields
October 30, 2015 Dr Yann Savoye #44
Elasto-Plastic Optimization
• Re-Weighted Iterative Subspace Solver
October 30, 2015 Dr Yann Savoye #45
Cage-based Template Registration
October 30, 2015 Dr Yann Savoye #46
Space-Time Output Surface
October 30, 2015 #47
Content
1. Introduction2. Background and Related Works3. Cage-based Inversion4. Cage-based Reskinning5. Cage-based Registration6. Cage-based Stylization7. Conclusions
Dr Yann Savoye
6
October 30, 2015 #48
Cage-based Stylization
Dr Yann Savoye
Cage-basedModeling
Cage-based Reskinning
Objective: Exaggerating surface motion by cage-based filtering.
Cage-basedRegistration
Cage-basedStylization
October 30, 2015 Dr Yann Savoye #49
Cage-based Shape Exaggeration
• Input Animation Mesh Sequence
• Cage-based Conversion
• Temporal Cage Parameters
• Cage-based Filtering
October 30, 2015 Dr Yann Savoye #50
Cage-based Shape Exaggeration
October 30, 2015 #51
Content
1. Introduction2. Background and Related Work3. Cage-based Inversion4. Cage-based Reskinning5. Cage-based Registration6. Cage-based Stylization7. Conclusions
Dr Yann Savoye
7
October 30, 2015 Dr Yann Savoye #52
• Capturing Life-Like Dynamic Geometry Into Cages.• A new Graphics-To-Vision Strategy built around the cage-based scheme.• Several Technical Tools in Shape Modeling, Animation, 4D Vision.
Conclusions
• Cage-based Registration / Filtering / Reskinning.• Cage-based Fitting And Non-Rigid ICP Scheme. • Unsupervized Correspondences and Watertight Point Cloud. • Silhouette-Awareness and Shape-Awareness.• Scalability / Interactivity / Linearity / Efficiency / Simplicity.
October 30, 2015 Dr Yann Savoye #53
Caged or not caged?
Even if shapes are enclosed inside cages, highly non-rigid deformation is allowed by a few degree of freedom – shapes are free at last!
October 30, 2015 #54
To add fuel to the fire…
Cage-based Performance CaptureStudies in Computational Intelligence – vol 509
Springer Book
Dr Yann Savoye
