1

Lecture 15-16 Pose Estimation – Gaussian Process

Tae-Kyun Kim

EE4-62 MLCV

2

MS KINECT

http://www.youtube.com/watch?v=p2qlHoxPioM

The KINECT body pose estimation is achieved by randomised regression forest techniques.

EE4-62 MLCV

3

We learn pose estimation as a regression problem, and Gaussian process as a cutting edge regression method.

EE4-62 MLCV

We see it through the case study (slide credits to): Semi-supervised Multi-valued Regression, Navaratnam, Fitzgibbon, Cipolla, ICCV07,

where practical challenges addressed are 1)multi-valued regression and 2)sparsity of data.

4

EE4-62 MLCV

A mapping function is learnt from the input image I to the pose vector , which is taken as a continuous variable.

θ

5EE4-62 MLCV

Given an image (top left) , e.g. a silhouette (top right) is obtained by background subtraction techniques (http://en.wikipedia.org/wiki/Background_subtraction). The estimated 3d pose is shown at two camera angels (bottom left and right).

6EE4-62 MLCV

We attempt to map 2D image space to 3D pose space. There is inherent ambiguity in pose estimation (as an example in the above).

7EE4-62 MLCV

Eye tracking can be tackled as a regression problem, where the input is an image I and the output is a eye location.

8EE4-62 MLCV

Typical image processing steps: Given an image, a silhouette is segmented. A shape descriptor is applied to the silhouette to yield a finite dimensional vector. (Belongie and Malik, Matching with Shape Contexts, 2000)

9EE4-62 MLCV

The output is a vector of m joint angles.

10EE4-62 MLCV

11EE4-62 MLCV

12EE4-62 MLCV

13EE4-62 MLCV

14EE4-62 MLCV

15

Gaussian Process

EE4-62 MLCV

16

EE462 MLCV

i.i.d. (independent identical distributed)

Review of Gaussian density estimation (lecture 1,2)

We want to find the Gaussian parameters from the given data. The problem is to find the parameters by maximising the posterior probability

𝑝 (𝜇 ,𝜎 2|𝐱 )=𝑝 (𝐱|𝜇 ,𝜎 2 )𝑝 (𝜇 ,𝜎2 )𝑝(𝐱 )

17

EE462 MLCV

Review of Gaussian density estimation (lecture 1,2)

We do not have priors on the parameters and data, thus we maximise the (log) likelihood function instead.

𝑝 (𝜇 ,𝜎 2|𝐱 )=𝑝 (𝐱|𝜇 ,𝜎 2 )𝑝 (𝜇 ,𝜎2 )𝑝(𝐱 )

Maximum Likelihood (ML) vs Maximum A Posterior (MAP) solutions:

P(X|Y)P(X|Y)P(Y) (e.g. Gaussian Process)

EE462 MLCVReview of polynomial curve fitting

(lecture 1,2)We want to fit a polynomial curve to given data pairs (x,t).

¿𝐰𝑇 𝐱= =

where

The objective ftn to minimise is

19

Gaussian ProcessesEE4-62 MLCV

20

EE4-62 MLCV

21

EE4-62 MLCV

𝐲 𝑁 (𝐲∨𝟎 ,𝐊 )

22

EE4-62 MLCV

𝐲 (𝐱)

𝐱

23

EE4-62 MLCV

Gaussian Processes for Regression

24

EE4-62 MLCV

25

EE4-62 MLCV

26

EE4-62 MLCV

27

EE4-62 MLCV

𝑡=𝑦 (𝑥 )+𝜖

28

EE4-62 MLCV

𝑁×𝑁𝑁×1

29

EE4-62 MLCV

30

31

EE4-62 MLCV

32

Gaussian Process Matlab Toolbox

http://www.lce.hut.fi/research/mm/gpstuff/install.shtml(try demo_regression_robust.m, demo_regression1.m)

33

Learning HyperparametersEE4-62 MLCV

34

35

Automatic Relevance DeterminationEE4-62 MLCV

36

37

38

Back to the pose estimation problem

39EE4-62 MLCV

40

• It’ll never work…

– is multivalued – and live in high dimensions

EE4-62 MLCV

41EE4-62 MLCV

42EE4-62 MLCV

43EE4-62 MLCV

44EE4-62 MLCV

45EE4-62 MLCV

46EE4-62 MLCV

47EE4-62 MLCV

GMM : Gaussian Mixture ModelGPLVM : Gaussian Process Latent Variable Model

48EE4-62 MLCV

𝑝 (𝜃|𝑧 )=𝑝 (𝜃 , 𝑧)𝑝 (𝑧)  

49EE4-62 MLCV

50EE4-62 MLCV

51EE4-62 MLCV

52EE4-62 MLCV

53EE4-62 MLCV

54EE4-62 MLCV

55EE4-62 MLCV

56EE4-62 MLCV

57EE4-62 MLCV

58EE4-62 MLCV

59EE4-62 MLCV

60EE4-62 MLCV

61EE4-62 MLCV

𝑝 (𝑧 ,𝜃 )= ∫𝑝 (𝑧 ,𝜃|𝑡 )𝑝 (𝑡 )𝑑𝑡

t: a continuous latent variable

62EE4-62 MLCV

63EE4-62 MLCV

64EE4-62 MLCV

65EE4-62 MLCV

66EE4-62 MLCV

67EE4-62 MLCV

68EE4-62 MLCV

69EE4-62 MLCV

70EE4-62 MLCV

71

Real-Time Human Pose Recognition in Parts from Single Depth Images (J. Shotton et al, 2011)

• Key features– Depth image as input– Real-time by Random Forest, and Part-based

EE4-62 MLCV

72

Progressive Search Space Reduction for Human Pose Estimation (Ferrari et al, 2008)

EE4-62 MLCV