3D Human Body Pose Estimation using GP-LVM Moin Nabi Computer Vision Group Institute for Research in...

3D Human Body Pose Estimation using GP-LVM

Moin Nabi

Computer Vision GroupInstitute for Research in Fundamental Sciences (IPM)

Introduction to Human Pose Estimation

Articulated pose estimation from single-view monocular image(s)

Application of Human Pose Estimation

■ Entertainment: Animation, Games■ Security: Surveillance■ Understanding: Gesture/Activity recognition

Difficulties of Human Pose estimation

■ Appearance/size/shape of people can vary dramatically

■ The bones and joints are observable indirectly (obstructed by clothing)

■ Occlusions■ High dimensionality of the state space■ Lose of depth information in 2D image projections

Difficulties of Human Pose estimation

■ Challenging Human Motion

Problem Backgrounds

■ Pose Estimation From Monocular Image

Goal: Reliable 3D Human Pose Estimation from single-camera input

Gaussian process

Gaussian process

Gaussian process

Gaussian process

Gaussian process

a 5x5 covariance matrix and a 3-d input vector was used to calculate the 2-d output mean vector and the corresponding variances

Gaussian process

Use for Regression

Linear Dimension Reduction

Linear Dimension Reduction

Find the best latent inputs by maximizing the marginal likelihood under the constraint that all visible variables must share the same latent values.

Linear Dimension Reduction

Find the best latent inputs by maximizing the marginal likelihood under the constraint that all visible variables must share the same latent values.

Linear Dimension Reduction

Linear Dimension Reduction

Linear Dimension Reduction

Gaussian process

Nonlinear Dimension Reduction

Nonlinear Dimension Reduction

Nonlinear Dimension Reduction

Nonlinear Dimension Reduction

Nonlinear Dimension Reduction

Nonlinear Dimension Reduction

Human Pose Estimationusing GP-LVM

Image -> Pose

In Latent Space

Human Pose Estimationusing GP-LVM

Motion capture example, representing 102-D data in 2-D

Human Pose Estimationusing GP-LVM

Result

Result

Pose from Action

Thank You

Pose from Action

Different Action has Different shape in latent space

Future Work

Guess Action from shape of model in latent space

Thank You