Utilizing Posterior Probability forRace-composite Age EstimationEarly Applications to MORPH-II

Benjamin Yip

NSF-REU in Statistical Data Mining and Machine Learningfor Computer Vision and Pattern Recognition

The University of North Carolina at Wilmington

Presentation Outline

1. Introduction

2. MORPH-II

3. Image Pre-processing

4. Methods

5. Preliminary Results

6. Conclusions

1

Introduction

The Importance of Age Estimation

The human face encodes a wealth of information about theindividual; when properly analyzed, this information becomesvaluable data that can be used in a wide array of applications [2]:

• Electronic Customer Relationship Management (ECRM)

• Surveillance monitoring• Biometrics

2

The Importance of Age Estimation

The human face encodes a wealth of information about theindividual; when properly analyzed, this information becomesvaluable data that can be used in a wide array of applications [2]:

• Electronic Customer Relationship Management (ECRM)• Surveillance monitoring

• Biometrics

2

The Importance of Age Estimation

The human face encodes a wealth of information about theindividual; when properly analyzed, this information becomesvaluable data that can be used in a wide array of applications [2]:

• Electronic Customer Relationship Management (ECRM)• Surveillance monitoring• Biometrics

2

Previous Research

Previous research [3] has shown that age estimation is highlysensitive to race and gender categories. However, as far as we aware,no previous models have taken multiracial individuals into account.

3

Project Overview

MORPH-IIPre-

processingGenderClsf.

Race ClassificationRace Classification

BM AgeEstimator

WM AgeEstimator

BF AgeEstimator

WF AgeEstimator

MaleFemale

Figure 1: The four subgroup age estimators correspond to: black females(BF), white females (WF), black males (BM) and white males (WM)

4

Utilizing Posterior Probability

Let b,w ∈ [0, 1] be the posterior probabilities of belonging to eachrace, where b+ w = 1.

Race Classification

Black AgeEstimator

White AgeEstimator

YB YW

b w

Y∗ = (b ∗ YB) + (w ∗ YW)

5

MORPH-II

Introduction to MORPH-II

Table 1: Number of Images by Gender and Race

Black White Asian Hispanic Other Total

Male 36,821 7,958 140 1,661 64 46,644Female 5,756 2,590 13 99 32 8,490Total 42,577 10,548 153 1,760 96 55,134

Table 2: Number of Distinct Individuals

Black White Asian Hispanic Other Total

Male 8829 2056 47 507 19 11458Female 1491 628 4 28 8 2159Total 10320 2684 51 535 27 13617

6

Introduction to MORPH-II

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

4.04

9 im

ages

0

1000

2000

3000

4000

5 10 15 20 25 35 50

Number of Images

Num

ber

of In

divi

dual

s

Images per Individual

Figure 2: Images per Subject

0

500

1000

1500

2000

15 20 25 30 35 40 45 50 55 60 65 70 75

Age

Fre

quen

cy

Age Distribution

Figure 3: Age Distribution

7

Image Pre-processing

Objectives of Pre-processing

Pre-processing is a very important operation in computer visiontasks, as valuable information is usually enveloped in meaninglessspace. Accordingly, the absence of a pre-processing step can havedrastic consequences on the results.

8

Pre-processing Objectives

Given a standardmugshot,

the objective is to:

1. ensure properrotation,

2. extract only theface, and

3. standardize theimage,

yielding thepre-processed face.

9

Pre-processing MORPH-II

Figure 4: Stages of Pre-processing Figure 5: Challenging Images

10

Methods

Subsetting

Given the heavily imbalanced nature of MORPH-II, only images ofBlack and White individuals were used in this project. These imageswere subsetted according to the below scheme:

bf_1 wf_1 bm_1 wm_1

bf_2 wf_2 bm_2 wm_2

f1

f2

m1

m2

Figure 6: Subsetting Scheme

In this reduced dataset there are a total of 20,560 images in a 3 to 1Male to Female ratio. The images for each gender are split into twohalves, and every race and gender subgroup is divided similarly (±1image). There are no common individuals between subsets.

11

Feature Extraction

After undergoing histogram equalization to correct for illuminationvariation, features were extracted from each image using LocalBinary Patterns (LBPs); these feature vectors were used for all stagesof the age estimation pipeline.

Figure 7: Examples of LBPs1

A Note on Dimensionality ReductionThe dimension of the LBP feature vectors was reduced using Principal ComponentAnalysis (PCA). 400 principal components were kept.1http://scikit-image.org/docs/0.12.x/auto_examples/features_detection/plot_local_binary_pattern.html

12

Race Classification

Race ClassifierSupport Vector Machine (SVM)with a linear kernel

• Used to obtain posteriorprobabilities

Table 3: Race Classification Accuracies

train : test Accuracy Cost

f1 : f2 96.03 .0002f2 : f1 95.97 .0001m1 : m2 96.01 .0001m2 : m1 98.13 .0001

Posterior probabilities are obtained by fitting a sigmoid/logisticmodel to the SVM outputs (Platt scaling) [4]:

P(y = 1 | f) = 11+ exp(Af+ B) (1)

13

Age Estimation

ModelSupport Vector Regression (SVR)with a linear kernel

Race Classification

Black AgeEstimator

White AgeEstimator

YB YW

b w

Y∗ = (b ∗ YB) + (w ∗ YW)

14

Preliminary Results

Partial Dataset

The Partial (even) dataset contains 1,000 images of 1,000 distinctindividuals; it differs from the reduced version of MORPH-II in thatthe overall age distribution is uniform.

Table 4: Results on Partial Dataset

MAE Weighted MAE

bf_1 6.71 6.695bf_2 6.634 6.45wf_1 7.007 6.535wf_2 6.579 6.461bm_1 5.501 5.475bm_2 4.979 4.799wm_1 4.443 4.434wm_2 4.518 4.498

15

Full Dataset (reduced)

Table 5: Results on Full Dataset (reduced)

MAE Weighted MAE

bf_1 6.894 6.872bf_2 6.316 6.389wf_1 5.826 5.824wf_2 5.791 5.863bm_1 5.283 5.279bm_2 5.359 5.362wm_1 4.985 4.983wm_2 4.971 4.987

16

Conclusions

Conclusions

While the results from the Partial dataset seem to show that thecomposite age estimate is an improvement over the straightforwardprediction, the preliminary results from the reduced MORPH-IIdataset are mixed. Much more work remains to be done in testingthe model on the full dataset.

17

Future Work

The race labels do not account for multiracial individuals; therefore,there may be mixed race individuals in the training sets that arediluting the models. To correct for this:

1. Assemble training sets of individuals who are not of mixed race(i.e. they fall clearly into the Black or White race categories)

2. Use these sets to train race classifiers and age models

18

Future Work

The race labels do not account for multiracial individuals; therefore,there may be mixed race individuals in the training sets that arediluting the models. To correct for this:

1. Assemble training sets of individuals who are not of mixed race(i.e. they fall clearly into the Black or White race categories)

2. Use these sets to train race classifiers and age models

18

References i

G. Bingham, B. Yip, M. Ferguson, and C. Nansalo.MORPH-II: Inconsistencies and Cleaning.

Y. Fu, G. Guo, and T. S. Huang.Age synthesis and estimation via faces: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence,32(11):1955–1976, Nov 2010.

G. Guo and G. Mu.Human age estimation: What is the influence across race andgender?In 2010 IEEE Computer Society Conference on Computer Visionand Pattern Recognition - Workshops, pages 71–78, June 2010.

References ii

J. C. Platt.Probabilistic outputs for support vector machines andcomparisons to regularized likelihood methods.In ADVANCES IN LARGE MARGIN CLASSIFIERS, pages 61–74. MITPress, 1999.

K. Ricanek and T. Tesafaye.Morph: A longitudinal image database of normal adultage-progression.In Automatic Face and Gesture Recognition, 2006. FGR 2006. 7thInternational Conference on, pages 341–345. IEEE, 2006.