Lec 15: Graph Embedding & Laplacianface · PCA is a special case of graph embedding oFully connected affinity map, equal importance LDA is a special case of graph embedding oFully

Spring 2020: Venu: Haag 315, Time: M/W 4-5:15pm

ECE 5582 Computer VisionLec 15: Graph Embedding & Laplacianface

Zhu LiDept of CSEE, UMKC

Office: FH560E, Email: [email protected], Ph: x 2346.http://l.web.umkc.edu/lizhu

Z. Li: ECE 5582 Computer Vision, 2020 p.1

slides created with WPS Office Linux and EqualX LaTex equation editor

Outline

Recap: Eigenface Fisherface HW-3: Fisherface and Laplacian face

Graph Embedding Laplacianface Graph Fourier Transform

Course project topics

Summary

p.2Z. Li: ECE 5582 Computer Vision, 2020

Subspace Learning for Face Recognition

Project face images to a subspace with basis A Matlab: x=faces*A(:,1:kd)

eigf1

eigf2

eigf3

= 10.9* + 0.4* + 4.7*


PCA & Fisher’s Linear Discriminant

• Between-class scatter

• Within-class scatter

• Where– c is the number of classes– i is the mean of class i

– | i | is number of samples of i..

Tii

c

iiBS ))((

1 --=å

=

å å= Î

--=c

i x

TikikW

ik

xS1

))((

1

2

1 2


Eigen vs Fisher Projection

p.5

• PCA (Eigenfaces)

Maximizes projected total scatter

• Fisher’s Linear Discriminant

Maximizes ratio of projected between-class to projected within-class scatter, solved by the generalized Eigen problem:

WSWW TT

WPCA maxarg=

WSW

WSWW

WT

BT

Wfld maxarg=

1 2

PCA

Fisher ��= ��

Z. Li: ECE 5582 Computer Vision, 2020

LDA implementation myLDA.m

p.6

% compute class meanmx = mean(x); ids = unique(y); m = length(ids);Sb = zeros(kd, kd); for k=1:m indx = find(y==ids(k)); nk(k) = length(indx); % class mean m_cx(k,:) = mean(x(indx, :)); % between class scatter Sb = Sb + nk(k)*(m_cx(k,:) - mx)'*(m_cx(k,:) - mx);end

% compute intra-class scatterSw = zeros(kd, kd); for k=1:m indx = find(y==ids(k)); nk(k) = length(indx); % remove mean xk = x(indx, :) - repmat(m_cx(k,:), [nk(k), 1]); % adding up Sw = Sw + (xk'*xk);end


LDA Implementation

Find projection by generalized Eigen problem solution

p.7

% generalized eigen problem[A, v]=eigs(Sb, Sw); if dbg figure(31); subplot(2,2,1); imagesc(Sb); colormap('gray'); title('S_b'); subplot(2,2,2); imagesc(Sw); colormap('gray'); title('S_w'); z = x*A; dist = pdist2(z, z); subplot(2,2,3); imagesc(dist); title('dist(j,k)'); subplot(2,2,4); stem(diag(v), '.'); grid on; hold on; title('eig v');end

��= ��


Fisherface Basis

It is interesting to compare Fisherface with Eigenface basis

p.8

FisherfaceEigenface


Fisherface Performance

Fisher vs Eigenface performance: (fisherface.m) 1200 face images, 144 subjects Eigen kd=32

p.9

144 subjectsROC


Outline

Recap: Eigenface Fisherface

Graph Embedding Locality Preserving Projection Laplacian Face Graph Fourier Transform

Summary


Locality Preserving Projection

Recall the dimension reduction formulations: find w, s.t y=wx: PCA:

LDA:

p.11

max�

wTSw

S = SB + SW


LPP Formulation – Affinity

To preserve local affinity relationship Affinity map (Similar to the Laplacian Eigen Map)

Selection of heat kernel size and threshold are important Hint: affinity matrix should be sparse

p.12

affinity histogram


LPP Formulation – Affinity Supervised

How to utilize the label info ? Heat map mapping is good for intra-class affinity modelling,

but how about intra-class affinity ? One direct solution is to set affinity to zero for intra class

pairs

p.13

% LPP - compute affinityf_dist1 = pdist2(x1, x1);% heat kernel sizemdist = mean(f_dist1(:)); h = -log(0.15)/mdist; S1 = exp(-h*f_dist1); id_dist = pdist2(ids, ids);subplot(2,2,3); imagesc(id_dist); title('label distance');S2=S1; S2(find(id_dist~=0)) = 0; subplot(2,2,4); imagesc(S1); colormap('gray'); title('affinity-supervised');


LPP- Affinity Preserving Projection

Find a projection that best preserves the affinity matrix

p.14

min�

�� −��

� , �. �. � =�X

min�

��X� −�X��

�

�. �, �� = �−exp��X� −X��

��ℎ , �X �X� −X�� ≤ �

0, �X��


LPP Formulation

L = D-S: nxn matrix, called graph Laplacian

Normalizing factor: nxn D Diagonal matrix, entry Dii = sum of col/row affinity The larger the value, the more important data point is

Constraint on D:


Generalized Eigen Problem

Now the formulation is,

Lagranian

by KKT (Karush-Khun-Tucker) Condition, it is solved by a generalized Eigen problem

p.16

��, �� = �� − ��X��−1�


X He, S Yan, Y Hu, P Niyogi, HJ Zhang, “Face Recognition Using Laplacianface”, IEEE Trans PAMI, vol. 27 (3), 328-340, 2005.

Matlab Implementation

laplacianface.m

p.17

%LPPn_face = 1200; n_subj = length(unique(ids(1:n_face)));

% eigenface kd=32; x1 = faces(1:n_face,:)*A1(:,1:kd); ids=ids(1:n_face);

% LPP - compute affinityf_dist1 = pdist2(x1, x1);% heat kernel sizemdist = mean(f_dist1(:)); h = -log(0.15)/mdist; S1 = exp(-h*f_dist1); figure(32); subplot(2,2,1); imagesc(f_dist1); colormap('gray'); title('d(x_i, d_j)');subplot(2,2,2); imagesc(S1); colormap('gray'); title('affinity'); %subplot(2,2,3); grid on; hold on; [h_aff, v_aff]=hist(S(:), 40); plot(v_aff, h_aff, '.-'); % utilize supervised infoid_dist = pdist2(ids, ids);subplot(2,2,3); imagesc(id_dist); title('label distance');S2=S1; S2(find(id_dist~=0)) = 0; subplot(2,2,4); imagesc(S1); colormap('gray'); title('affinity-supervised');

% laplacian facelpp_opt.PCARatio = 1; [A2, eigv2]=LPP(S2, lpp_opt, x1);


Laplacian Face

Now, we can model face as a LPP projection:

p.18

Eigenface Laplacian Face


Laplacian vs Eigenface

1200 faces, 144 subjects


LPP and PCA Graph Embedding is an unifying theory on dimension

reduction PCA becomes special case of LPP, if we do not enforce local

affinity


LPP and LDA

How about LDA ? Recall within class scatter:

p.21

This is i-th classData covariance

Li has diagonal entry of 1/ni,Equal affinity among data points


LPP and LDA

Now consider the between class scatter C is the data covariance,

regardless of label L is graph Laplacian computed

from the affinity rule that,

p.22

��, �� = �1��

, �X X�, X��X� XX G �X��

0, �X��


LDA as a special case of LPP

The same generalized Eigen problem


Graph Embedding Interpretation of PCA/LDA/LPP

Affinity graph S, determines the embedding subspace W, via

PCA and LDA are special cases of Graph Embedding PCA:

LDA

LPP

p.24

�� = �−exp��X� −X��

��ℎ , �X �X� −X�� ≤ �

0, �X��

�� = �1��

, �X X�, X� ∈ ��

0, �X��

�� = 1/�


Applications: facial expression embedding

Facial expressions embedded in a 2-d space via LPP

p.25

frown

sad

happy

neutral


Application: Compression of SIFT

Compression of SIFT, preserve matching relationship, rather than reconstruction:


� =argmin�

��X� −�X��

�

Homework-3: Subspace Methods

Objective: Understand the graph embedding connections among popular

subspace methods like PCA, LDA and LPP Practical experiences with serious size data set

Data Set: https://umkc.app.box.com/s/0qu7tc3jb88at2h53l1dpcuqkt9pn

7ww 417 subjects, 6650 image face data set, pre-processed to

20x20 pel images, intensity normalized to [0, 1] Add your own face images, 10~15, frontal

Tasks: Compute Eigenface, Fisherface and Laplacianface models ROC plot on verification performance mAP for retrieval/identification performance


HW-3 test run

Laplacian face is powerful.


Graph Fourier Transform David I. Shuman, Sunil K. Narang, Pascal Frossard, Antonio Ortega, Pierre Vandergheynst:

The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains. IEEE Signal Process. Mag. 30(3): 83-98 (2013)


Signal on Graph

non-uniformly sampled


Graph Fourier Transform

GFT is different from Laplacian Embedding:


GFT Example

Graph Laplacian


Normalized Graph Laplacian

Normalize by edge pair degree


Graph Frourier Transform

Analogous to FT


Graph Spectrum

Freq interpretation


GetGFT.m

Implementation of GFT: affinity graph sparsity control is important


Graph Signal Smoothness

Quadratic form on L:


most smooth: low freq dominating

Summary

Graph Laplacian Embedding is an unifying theory for feature space dimension reduction PCA is a special case of graph embedding

o Fully connected affinity map, equal importance LDA is a special case of graph embedding

o Fully connected intra classo Zero affinity inter class

LPP: preserves pair wise affinity. GFT: eigen vectors of graph Laplacian, has Fourier Transform

like characteristics. Many applications in Face recognition Pose estimation Facial expression modeling Compression of Graph signals.


Documents

Lec 15: Graph Embedding & Laplacianface · PCA is a special case of graph embedding oFully connected affinity map, equal importance LDA is a special case of graph embedding oFully