8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

1/26

Using a sparse learning Relevance VectorMachine in Facial Expression Recognition

Dragos Datcu

April 14, 2006

Euromedia 2006, Dragos Datcu

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

2/26

Introduction

The utility of facial expression recognition technology:

Human computer interfaces

Safety, surveillance- terorist identification- safe driving, somnolence detection- access control

Psychology

Euromedia 2006, Dragos Datcu 1

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

3/26

Problem definition

How to realize a fully automatic facial expression recognition system

using a sparse learning Relevance Vector Machine?

The automatic facial expression recognition system includes:

- face detector- facial feature extractor for mouth, left and right eye- Facial Characteristic Point - FCP extractor- facial expression recognizer

Euromedia 2006, Dragos Datcu 2

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

4/26

Facial expression recognition system

Euromedia 2006, Dragos Datcu 3

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

5/26

I. Face detection

The method makes use of:

- Viola&Jones features (24x24 size samples, 162336 features/sample)- Evolutionary AdaBoost (150 size population)

- Relevance Vector Machine (RVM) - weak classifier

- training data set includes: 4916 faces and 10000 non-faces.

The detector consists in a 32 layer cascade of classifiers using 4297 features

Euromedia 2006, Dragos Datcu 4

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

6/26

Viola&Jones features

The basic types:

Applied on an image:

The value is: Haar value =P

PixelintensitydarkareasP

Pixelintensitylightareas

For a 24x24 image, there exist more than 160.000 such features.

Euromedia 2006, Dragos Datcu 5

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

7/26

AdaBoost algorithm

Discrete AdaBoost [Freund and Schapire (1996b)]

1. Start with weights wi = 1/N, i = 1,...,N.

2. Repeat for m = 1, 2,...,M:

(a) Fit the classifier fm(c) {1, 1} using weights wi on the training data.(b) Compute errm = Ew[1(y=fm(x))], cm = log((1 errm)/errm).(c) Set wi wiexp[cm1(yi=fm(xi))], i = 1, 2,...,N, and renormalize so thatP

i wi = 1.

3. Output the classifier sign[PM

m=1 cmfm(x)].

Euromedia 2006, Dragos Datcu 6

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

8/26

The RVM-based weak classifier

Euromedia 2006, Dragos Datcu 7

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

9/26

Evolutionary AdaBoost

E.A. performs an efficient search for the representative Viola&Jonesfeatures for classification

Euromedia 2006, Dragos Datcu 8

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

10/26

Face detection

Euromedia 2006, Dragos Datcu 9

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

11/26

Face detection

Euromedia 2006, Dragos Datcu 10

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

12/26

Face detection

Cascaded classifier with T layers

Euromedia 2006, Dragos Datcu 11

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

13/26

Face detection

Example: choosing the propper weak classifier for three different V&Jfeatures.2-fold cross validation results on three week classifiers for face detection based on Haar-like

features

Euromedia 2006, Dragos Datcu 12

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

14/26

Face detection

ROC curves of three kernels, obtained by adjusting each classifiers threshold

Euromedia 2006, Dragos Datcu 13

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

15/26

Face detection results

RVM test results, both training and testing are performed on MIT CBCL database

Euromedia 2006, Dragos Datcu 14

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

16/26

Face detection results

RVM test results, the training is done using MIT CBCL, the testing is done on CMU

database

Euromedia 2006, Dragos Datcu 15

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

17/26

II. Facial feature extraction

The facial features to be extracted are: left/right eye and mouth areas.

Euromedia 2006, Dragos Datcu 16

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

18/26

III. FCP model

Kobayashi and Hara model the face through 30 FCPs

There are three steps involved in the FCP detection:

1. FCP detection using corner detectors2.a. FCP detection using RVM classifier

2.b. FCP detection using integral projection method

Euromedia 2006, Dragos Datcu 17

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

19/26

III.1. FCP detection using corner detectors

There are two corner detectors that are used as a first stage for FCP detection.

The hybrid corner detector stands for a combination of two corner detectors:

- Harris

- Sojka

Euromedia 2006, Dragos Datcu 18

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

20/26

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

21/26

III.2.a. The FCPs to be extracted with RVM based E.A.

classifier

Euromedia 2006, Dragos Datcu 20

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

22/26

III.2.a. The stages of training the FCP detector

Euromedia 2006, Dragos Datcu 21

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

23/26

III.2.a. FCP data set

BioID dataset and the Carnegie Melon dataset

Euromedia 2006, Dragos Datcu 22

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

24/26

III.2.a. EA characteristics

Euromedia 2006, Dragos Datcu 23

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

25/26

III.2.a. FCP detection results

Euromedia 2006, Dragos Datcu 24

8/14/2019 Using a sparse learning Relevance Vector Machine in Facial Expression Recognition

26/26

III.2.b. FCP detection, integral projection method

It is used to extract the rest of the points.

- projects the image into the vertical and horizontal axes

- obtain the boundaries

- the boundaries of the features have relatively high contrast

- the image is presented by two 1D orthogonal projection functions: IP Fv

, IP Fh

,

M I P F v, M I P F h, V IP Fv, V IP Fh, GP Fv, GP Fh

Euromedia 2006, Dragos Datcu 25