Prediction of Non-Linear Aging Trajectories of Faces

Prediction of Non-Linear Aging Trajectories of Faces

K. Scherbaum, M. Sunkel, V. Blanz and H.-P. Seidel

[ 2007/5/9, Eurographics 2007, Prague ]

Kristina Scherbaum [email protected]

Motivation / Goal

automated growth-prediction system applications

photofit-pictures of missing children automated animation, art


9 years

10 years 11 years

Age Progression – Optimal Case

face space

child 1

child 2

child 3


Real Case – Support Vector Regression

face space

9 years

10 years 11 years

only 1 sample per person no longitudinal study

find isosurfaces and gradients

Runge-Kutta Integration

9 years

10 years

11 years


Main Assumption - Curved Trajectories

use machine learning non-linear Support Vector Regression

integration of local age-gradient

growing faces transform along curved trajectories


Challenges

learn change over time of individual facesnon-linear dependency on time, curved trajectory

learn how the change depends on individual facenon-linear dependency in face space

sparse dataset, no longitudinal study


3D Morphable Facemodel

System is based on a Morphable 3D Facemodel [Blanz,Vetter‘99] Built from 200 3D-face-scans of adults


3D Morphable Facemodel

vector space of faces

vectors with point-to-point correspondence

Shape

Texture


Representation of Faces - Face Spaces

PCA to reduce dimensionality (yields coefficients)


Extended Morphable Model

Extension by … plus ~238 facemodels of teenagers 3 simultaneous laser scans per face

Correspondence by … top-down approach fitting Morphable Model to new 3D faces merging original data and best fit


Fitting the Morphable Model to 3D Scans

no optical flow because scans are often incomplete

best fit of the morphable model

merged result3D laser scans


learn function that maps any face x to a scalar age y

to learn this function we use …non-linear Support-Vector-Regression on training sets of l pairs

Age Progression Algorithm1


Fitting a Regression Curve

for a given set of samples find f(x) such that all samples are within an -tube preselect and tradeoff between smoothness and errors of outliers

2

x

y

Linear: f(x) = wx + b Non-linear: f is sum of Gaussian RBF kernels K(x-xi)


Gaussian RBF (Radial Basis Function) as kernel

we applied grid search using cross validation to optimize parameters such as (Kernelwidth) i and b are determined by SVM training

using LIBSVM for -Support Vector Regression

Non-Linear SVM Regression2


Isosurfaces are defined in PCA space Gradient gives shortest path to next isosurface

Along the gradient … many facial changes due to aging almost no other changes

(known technique, Blanz et al. 99)

Thus: Compute growth along the gradient!

Local Aging3


Gradient Example - Facial Attributes

gender manipulation

original

3

femalemale


Growth Simulation: New Approach3

growth curve with given face x0 at time t currently we compute the local gradient and walk along this gradient instead we should compute the curved trajectory


Solve differential equation … to compute curved trajectories integrate the differential equationusing Runge-Kutta algorithmperform small steps

Runge Kutta Integration4


Visualized Aging Trajectories4


Reducing Complexity

we did not train on all principle components speedup of SVM training we experimented with 20, 40 or 80 PCs

Justification …

growth leads to overall change of facial size significant changes are represented

by the first PCs [ large variance ] facial growth should happen in the first PCs

4


Growth Example

growth simulation for both, shape and texture

12 14 16 18 20 years

22 24 26 28 30 years


More Examples

10 years

15 years

20 years

30 years

12 13 12 10 14

3D laser scans,original

age


Background, Haircut

Face

Pose, Light

3D reconstruction and aging

Rendering the Result into Images [EG’04]

Composed Result


Input at the age of 11

Photofit Picture Example

Possible appearances at the age of 17


Aging in Images - Example

Picture (1999)

Ground truth pictures (2005)

Different prediction renderings

3D reconstruction and aging


Linear vs. Non-Linear

Linear age progression perform linear regression (yields a function)

[ straight-forward least squares fit ] transform faces also along the gradient

Disadvantages … the gradient is constant [ linear function ] each face moves along

the same straight trajectory


Linear vs. Non-Linear

comparison of age estimation error (in months) mean squared training and generalization errors

non-linear (RBF)

38.90 29.35

linear 67.66 62.87

non-linear (RBF)

32.68 18.12

linear 66.14 60.05

non-linear SVM regression behaves superior! generalization indicates: no overfitting


Remember the Challenges

Are growth trajectories curved?Mean angle between start- and target-tangent

the trajectories are curved, not linear

10.3º 30.0º

Have different faces distinct trajectories?Mean angle of trajectories of different faces

15.7º 33.5º

the trajectories are different


Conclusions

Results … aging involves non-linear components trajectories are distinct for different individuals linear systems are a reasonable approximation technique works without longitudinal data

But … more data would be helpful longitudinal data would allow for exact evaluation


MOVIE

Thank you for your attention!


Representation of Faces - Face Spaces

arbitrary faces by linear combinations of examples

PCA to reduce dimensionality (yields coefficients)


Main Idea … compute aging trajectories z(t) locally along gradient of the aging function f(x) and going through a start vector or face x0:

Aging Trajectories4


Aging Information

extracted from the database of 200 adult face scans and new database of 238 face scans of teenagers

teenager overview


Documents

Prediction of Non-Linear Aging Trajectories of Faces