ECE738 Advanced Image Processing

Face Recognition by Elastic Bunch Graph Matching

IEEE Trans. PAMI, July 1997

(C) 2005 by Yu Hen Hu 2ECE738 Advanced Image Processing

(C) 2005 by Yu Hen Hu 3ECE738 Advanced Image Processing

(C) 2005 by Yu Hen Hu 4ECE738 Advanced Image Processing

Gabor Transform

• Gabor Function

2 22 20 0

0 0

( , ) exp(

exp 2

G x y x x a y y b

j u x x v y y

Daugman, IEEE Trans. ASSP July 1988

(C) 2005 by Yu Hen Hu 5ECE738 Advanced Image Processing

Gabor Wavelet Transform

An implementation of Gabor transform

Gaussian envelop width = 2Last term in complex sinusoids removes DC in the kernel

5 level spatial frequency from 4 to 16 pixels in an 128 x 128 image, 8 orientations

2 2 2 2 21 1 2 2

2 2

2

( )( ) exp

2

exp( ) exp2

k k x k xx

ik x

Daugman, IEEE Trans. ASSP July 1988

(C) 2005 by Yu Hen Hu 6ECE738 Advanced Image Processing

Jeta set of 40 (5 spatial frequency, 8 orientations) complex Gabor wavelet coefficients for one image point.

J = [a1, a2, …, a40]

Similarity between jets:

d is the displacement of pixels: needs to be estimated.kj: spatial wave vector

' '

, ' ' '

cos, '

'

Ta

jj j j jj

S J J J J J J

a a d kS J J

J J

Fig. 1. Similarities Sa(J,J’) (dashed line) and S(J,J’) (solid line) with J’ taken from the left eye of a face, and J taken from pixel positions of the same horizontal line. The dotted line shows the estimated displacement d (divided by eight to fit the ordinate range). The right eye is 24 pixels away from the left eye, generating a local maximum for both similarity functions and zero displacement close to dx = -24.

(C) 2005 by Yu Hen Hu 7ECE738 Advanced Image Processing

Face Graph

• Facial fiducial points– Pupil, tip of mouth, etc.

• Face graph– Nodes at fiducial pts.– Un-directed graph– Object-adaptive– The structure of graph is the

same for each face– Fitting a face image to a face

graph is done automatically – Some nodes may be

undefined due to occlusion. Hence, association of nodes of different face graphs may need to be done manually.

• Bunch– A set of Jets all asso with

the same fiducial pt.– e.g. an eye Jet may consists

of different types of eyes: open, closed, male, female, etc.

• Face bunch graph (FBG): – Same as a face graph,

except each node consists of a jet bunch rather than a jet

(C) 2005 by Yu Hen Hu 8ECE738 Advanced Image Processing

Face Bunch Graph

• Has the same structure as individual face graph

– Each node labeled with a bunch of jets

– Each edge labeled with average distance between corresponding nodes in face samples

• Given a new face, an elastic bunch graph matching (EBGM) method selects the best fitting jets (local experts) from the bunch dedicated to each node in the face bunch graph.

/B Bme emx x M

(C) 2005 by Yu Hen Hu 9ECE738 Advanced Image Processing

Elastic Bunch Graph Matching

Graph similarity measure

: weighting factor

• Initially, manually generate a few FGs to create a FBG

• Heuristic algorithm to find the image graph that maximizes the similarity:– Coarse scan of image using

jets to detect face

– Varying sizes and aspect ratio of FBG to adapt to right format of face.

– Finally, all nodes are moved locally to maximize SB.

2

2

1( , ) max ,

: displacement on edge e

: jet at node n

I I BmB n n

mn

I Be e

Bee

Ie

In

S G B S J JN

x x

E x

x

J

(C) 2005 by Yu Hen Hu 10ECE738 Advanced Image Processing

Results