March 1, 2002 1ICT Virtual Human Workshop
HUMAN FIGURE ANIMATION
Norman I. BadlerNorman I. Badler
Center for Human Modeling and SimulationCenter for Human Modeling and Simulation
University of PennsylvaniaUniversity of Pennsylvania
Philadelphia, PA 19104-6389 USAPhiladelphia, PA 19104-6389 USA
http://www.cis.upenn.edu/~badlerhttp://www.cis.upenn.edu/~badler
Norman I. BadlerNorman I. Badler
Center for Human Modeling and SimulationCenter for Human Modeling and Simulation
University of PennsylvaniaUniversity of Pennsylvania
Philadelphia, PA 19104-6389 USAPhiladelphia, PA 19104-6389 USA
http://www.cis.upenn.edu/~badlerhttp://www.cis.upenn.edu/~badler
March 1, 2002 2ICT Virtual Human Workshop
Outline
• Generative Methods and TradeoffsGenerative Methods and Tradeoffs• Re-use and RetargettingRe-use and Retargetting• Parametric Models and ControlsParametric Models and Controls• Real-time or Offline? (i.e., Hard problems)Real-time or Offline? (i.e., Hard problems)• Model AvailabilityModel Availability• Body Requirements and AssumptionsBody Requirements and Assumptions• Object – Virtual Human InteractionsObject – Virtual Human Interactions
• Generative Methods and TradeoffsGenerative Methods and Tradeoffs• Re-use and RetargettingRe-use and Retargetting• Parametric Models and ControlsParametric Models and Controls• Real-time or Offline? (i.e., Hard problems)Real-time or Offline? (i.e., Hard problems)• Model AvailabilityModel Availability• Body Requirements and AssumptionsBody Requirements and Assumptions• Object – Virtual Human InteractionsObject – Virtual Human Interactions
March 1, 2002 3ICT Virtual Human Workshop
The “Realism Ceiling” for Human Models
Visual RealismVisual Realism
Time toTime tocreatecreate
BehavioralBehavioral RealismRealism
SpecialSpecialeffectseffects
InanimateInanimateobjectsobjects
Life-likeLife-likevirtualvirtual
humanshumans
Real-timeReal-time AgentsAgents
March 1, 2002 4ICT Virtual Human Workshop
What Approaches Push the Curve Toward More Realism?
Visual RealismVisual RealismInanimateInanimateobjectsobjects
MotionMotionCaptureCapture
ParameterizationParameterization(face, gait, gesture)(face, gait, gesture)
Life-likeLife-likevirtualvirtual
humanshumans
Time toTime tocreatecreate
BehavioralBehavioral RealismRealism
Real-timeReal-time
March 1, 2002 5ICT Virtual Human Workshop
Why is Realism Still Hard?
Visual RealismVisual RealismInanimateInanimateobjectsobjects
MotionMotionCaptureCapture
ParameterizationParameterization(face, gait, gesture)(face, gait, gesture)
Difficultto
generalize
What are the“right”
parameters?
Life-likeLife-likevirtualvirtual
humanshumans
Time toTime tocreatecreate
BehavioralBehavioral RealismRealism
Real-timeReal-time
March 1, 2002 6ICT Virtual Human Workshop
ProceduresProcedures Use models;Use models;
Generalize control;Generalize control;
FastFast
Find models;Find models;
Specific code;Specific code;
Integration w/othersIntegration w/others
Motion Motion CaptureCapture
Natural;Natural;
Nuance;Nuance;
PhysicsPhysics
Actors; Acting; Actors; Acting;
Motion specificity;Motion specificity;
RetargettingRetargetting
DynamicsDynamics
(Physics-(Physics-basedbased
simulation)simulation)
Physics;Physics;
Apparent interlimb Apparent interlimb coordination;coordination;
Free fall (rag doll)Free fall (rag doll)
Control laws;Control laws;
Model complexity;Model complexity;
RetargettingRetargetting
Collision responseCollision response
Trade-Offs Advantages Disadvantages
March 1, 2002 7ICT Virtual Human Workshop
Human Movement Categories
• Voluntary (task-oriented, reach, look-at)Voluntary (task-oriented, reach, look-at)
• Involuntary (breathing, balancing, blinking)Involuntary (breathing, balancing, blinking)
• SubconsciousSubconscious– Low level motor functions (fingers, legs, lips)Low level motor functions (fingers, legs, lips)
– Communicative acts (facial expressions, limb Communicative acts (facial expressions, limb gestures, body posture) gestures, body posture)
What generative techniques work for these?What generative techniques work for these?
• Voluntary (task-oriented, reach, look-at)Voluntary (task-oriented, reach, look-at)
• Involuntary (breathing, balancing, blinking)Involuntary (breathing, balancing, blinking)
• SubconsciousSubconscious– Low level motor functions (fingers, legs, lips)Low level motor functions (fingers, legs, lips)
– Communicative acts (facial expressions, limb Communicative acts (facial expressions, limb gestures, body posture) gestures, body posture)
What generative techniques work for these?What generative techniques work for these?
March 1, 2002 8ICT Virtual Human Workshop
Human Movement Implementations
• Voluntary Voluntary (procedures [IK], mocap)(procedures [IK], mocap)
• Involuntary Involuntary (procedures [physiological or dynamic (procedures [physiological or dynamic models])models])
• Low level motor functions Low level motor functions (procedures, mocap, (procedures, mocap, dynamics)dynamics)
• Communicative acts Communicative acts (mocap, procedures)(mocap, procedures)
category X technique = {animation paper}category X technique = {animation paper}
• Voluntary Voluntary (procedures [IK], mocap)(procedures [IK], mocap)
• Involuntary Involuntary (procedures [physiological or dynamic (procedures [physiological or dynamic models])models])
• Low level motor functions Low level motor functions (procedures, mocap, (procedures, mocap, dynamics)dynamics)
• Communicative acts Communicative acts (mocap, procedures)(mocap, procedures)
category X technique = {animation paper}category X technique = {animation paper}
March 1, 2002 9ICT Virtual Human Workshop
MoCap Retargetting
• Joint angles don’t neatly map to different Joint angles don’t neatly map to different size or structured figures.size or structured figures.• Need to maintain constraints (IK) to avoid Need to maintain constraints (IK) to avoid feet slipping and sliding.feet slipping and sliding.• Need to add IK to change reach and look-at Need to add IK to change reach and look-at direction.direction.• Moving interactions may fit but change Moving interactions may fit but change dynamics (e.g., figure skating pairs).dynamics (e.g., figure skating pairs).
• Joint angles don’t neatly map to different Joint angles don’t neatly map to different size or structured figures.size or structured figures.• Need to maintain constraints (IK) to avoid Need to maintain constraints (IK) to avoid feet slipping and sliding.feet slipping and sliding.• Need to add IK to change reach and look-at Need to add IK to change reach and look-at direction.direction.• Moving interactions may fit but change Moving interactions may fit but change dynamics (e.g., figure skating pairs).dynamics (e.g., figure skating pairs).
March 1, 2002 10ICT Virtual Human Workshop
Parametric Models (Procedures)
• IKIK
• GaitGait
• FacesFaces
• EyesEyes
• Muscles (Deformations)Muscles (Deformations)
• Gestures (BEAT/MagiCster)Gestures (BEAT/MagiCster)
• EMOTE EMOTE
• IKIK
• GaitGait
• FacesFaces
• EyesEyes
• Muscles (Deformations)Muscles (Deformations)
• Gestures (BEAT/MagiCster)Gestures (BEAT/MagiCster)
• EMOTE EMOTE
March 1, 2002 11ICT Virtual Human Workshop
IK at Work
Eye view;Note attentionControl
Instructions translated to Instructions translated to parameterized actions.parameterized actions.
March 1, 2002 12ICT Virtual Human Workshop
Walking (Sun and Metaxas)
Gait from parameterized biomechanical Gait from parameterized biomechanical model, original movements from motion model, original movements from motion capture and interpolated based on terrain capture and interpolated based on terrain angle.angle.
moviemovie
Gait from parameterized biomechanical Gait from parameterized biomechanical model, original movements from motion model, original movements from motion capture and interpolated based on terrain capture and interpolated based on terrain angle.angle.
moviemovie
March 1, 2002 13ICT Virtual Human Workshop
Gesture Control: EMOTE
• EMOTE: A real-time motion modification EMOTE: A real-time motion modification system.system.
• Defines Defines qualitiesqualities of movement with 8 of movement with 8 parameters.parameters.
• Based on Effort and Shape components of Based on Effort and Shape components of Laban Movement Analysis.Laban Movement Analysis.
• EMOTE: A real-time motion modification EMOTE: A real-time motion modification system.system.
• Defines Defines qualitiesqualities of movement with 8 of movement with 8 parameters.parameters.
• Based on Effort and Shape components of Based on Effort and Shape components of Laban Movement Analysis.Laban Movement Analysis.
March 1, 2002 14ICT Virtual Human Workshop
Inputs EMOTE Output
Key PosesKey Poses
End EffectorEnd EffectorGoalsGoals
Motion CaptureMotion Capture
ProceduresProcedures
Frame Frame RateRatePosesPoses
4 Efforts 4 Efforts
4 Shapes4 Shapes
InverseInverseKinematicsKinematics
InterpolationInterpolation
March 1, 2002 15ICT Virtual Human Workshop
Motion Qualities are Important
• Movements with EMOTE qualities give Movements with EMOTE qualities give insight into the agent’s cognitive state.insight into the agent’s cognitive state.
• When EMOTE qualities spread from limbs When EMOTE qualities spread from limbs to body, movements appear more sincere.to body, movements appear more sincere.
• Movements with EMOTE qualities give Movements with EMOTE qualities give insight into the agent’s cognitive state.insight into the agent’s cognitive state.
• When EMOTE qualities spread from limbs When EMOTE qualities spread from limbs to body, movements appear more sincere.to body, movements appear more sincere.
March 1, 2002 16ICT Virtual Human Workshop
Don’t Animators Know This?
Good key pose animators know this and use Good key pose animators know this and use character deformation to great advantage;character deformation to great advantage;
EMOTE allows this concept to be applied EMOTE allows this concept to be applied parametrically (under program or interactive parametrically (under program or interactive control) to articulated (jointed) figures.control) to articulated (jointed) figures.
Good key pose animators know this and use Good key pose animators know this and use character deformation to great advantage;character deformation to great advantage;
EMOTE allows this concept to be applied EMOTE allows this concept to be applied parametrically (under program or interactive parametrically (under program or interactive control) to articulated (jointed) figures.control) to articulated (jointed) figures.
March 1, 2002 17ICT Virtual Human Workshop
An Actor Trying to Act (Apologies to W. Shakespeare)
We’ll manually match these movements inWe’ll manually match these movements inour virtual human model, then vary theour virtual human model, then vary themotion qualities via EMOTE.motion qualities via EMOTE.
March 1, 2002 22ICT Virtual Human Workshop
With Light and Sustained Efforts (More solemn and serious?)
March 1, 2002 24ICT Virtual Human Workshop
Hit the ball
… forcefully.
…softly.
Manner Variants (adverbs): HIT (with real dynamic response)
March 1, 2002 25ICT Virtual Human Workshop
DI-GuyTM – Being combined with EMOTE (mocap + parameters)
March 1, 2002 26ICT Virtual Human Workshop
Procedural Eye Movement Model from Human Performance Data
SourceSource
Eyes fixedEyes fixedaheadahead
Eyes movedEyes movedby statisticalby statistical
modelmodel
Full MPEG-4Full MPEG-4faceface
March 1, 2002 27ICT Virtual Human Workshop
Important Problems to Solve in Real-Time• Constrained reach planning/executionConstrained reach planning/execution
• Coordination and consistencyCoordination and consistency
• Dynamics on demand (integrated with Dynamics on demand (integrated with constraints/goals)constraints/goals)
• Individuation (and its specification)Individuation (and its specification)
• Collision responseCollision response
• Deformable surfaceDeformable surface
• ClothingClothing
• Constrained reach planning/executionConstrained reach planning/execution
• Coordination and consistencyCoordination and consistency
• Dynamics on demand (integrated with Dynamics on demand (integrated with constraints/goals)constraints/goals)
• Individuation (and its specification)Individuation (and its specification)
• Collision responseCollision response
• Deformable surfaceDeformable surface
• ClothingClothing
March 1, 2002 29ICT Virtual Human Workshop
Model Availability
• ““Standard” H-Anim (seamless) avatars Standard” H-Anim (seamless) avatars (VRML / WEB3D / MPEG-4)(VRML / WEB3D / MPEG-4)• DI-Guy (Joint angle control)DI-Guy (Joint angle control)• Jack (Toolkit API; plug-ins)Jack (Toolkit API; plug-ins)• Application-specific models (Extempo, Application-specific models (Extempo, Ananova, …)Ananova, …)• Custom models in Maya, 3DSMax, …Custom models in Maya, 3DSMax, …
Issues: Support, Platform, API, Body SwapIssues: Support, Platform, API, Body Swap
• ““Standard” H-Anim (seamless) avatars Standard” H-Anim (seamless) avatars (VRML / WEB3D / MPEG-4)(VRML / WEB3D / MPEG-4)• DI-Guy (Joint angle control)DI-Guy (Joint angle control)• Jack (Toolkit API; plug-ins)Jack (Toolkit API; plug-ins)• Application-specific models (Extempo, Application-specific models (Extempo, Ananova, …)Ananova, …)• Custom models in Maya, 3DSMax, …Custom models in Maya, 3DSMax, …
Issues: Support, Platform, API, Body SwapIssues: Support, Platform, API, Body Swap
March 1, 2002 30ICT Virtual Human Workshop
Body Requirements and Assumptions
• Enough joints, correct placement, DOFs, limitsEnough joints, correct placement, DOFs, limits
• Skin surface shape (deformations)Skin surface shape (deformations)
• Torso shape and flexibilityTorso shape and flexibility
• Physiology: breathing, itching, eating, …Physiology: breathing, itching, eating, …
• Rendering: skin, hair, moisture, vascular changesRendering: skin, hair, moisture, vascular changes
• Human or caricature?Human or caricature?
• Resemblance to someone (or not)Resemblance to someone (or not)
• ClothingClothing
• Enough joints, correct placement, DOFs, limitsEnough joints, correct placement, DOFs, limits
• Skin surface shape (deformations)Skin surface shape (deformations)
• Torso shape and flexibilityTorso shape and flexibility
• Physiology: breathing, itching, eating, …Physiology: breathing, itching, eating, …
• Rendering: skin, hair, moisture, vascular changesRendering: skin, hair, moisture, vascular changes
• Human or caricature?Human or caricature?
• Resemblance to someone (or not)Resemblance to someone (or not)
• ClothingClothing
March 1, 2002 31ICT Virtual Human Workshop
Object – Human Interactions: Challenges
• Grasping (one / two hands, or with other Grasping (one / two hands, or with other body parts)body parts)
• Multi-person actions (carrying furniture)Multi-person actions (carrying furniture)
• Smart objects; Compliant motion Smart objects; Compliant motion (constraints applied to existing motions)(constraints applied to existing motions)
• Knowledge of how things operate, fit Knowledge of how things operate, fit together, disassembletogether, disassemble
• Grasping (one / two hands, or with other Grasping (one / two hands, or with other body parts)body parts)
• Multi-person actions (carrying furniture)Multi-person actions (carrying furniture)
• Smart objects; Compliant motion Smart objects; Compliant motion (constraints applied to existing motions)(constraints applied to existing motions)
• Knowledge of how things operate, fit Knowledge of how things operate, fit together, disassembletogether, disassemble
March 1, 2002 32ICT Virtual Human Workshop
The Agent needs to Know …
Action choices causeAction choices causeundesirable effectsundesirable effects
Actions changedActions changedto prevent errorsto prevent errors