Principal Component Analysis

Principal Component Analysis

Objective: Project the data onto a lower dimensional space while minimizing the information loss

Principal Component Analysis

load mnistm = mean(data);for i=1:size(data_m,2) data_m(:,i) = data(:,i) - m(i);end[pc,evals] = pca_OF(data_m); pc_data = data_m*pc(1:200,:)';

Principal Component Analysis

function [pc,evals] = pca_OF(x)[pc,evals] = eig(cov(x));evals = diag(evals);[evals, si] = sort(-evals); %Sort eigenvaluesevals = -evals;pc = pc(:,si)'; %Sort eigenvectors by magnitude of eigenvalues

Sorted Eigenvalues

Normalized Cumulative Variance (information preserved)

Projecting the digits onto the first two PCs

Projecting the digits onto PCs 1 and 3

Projecting the digits onto PCs 2 and 3

Recognition Accuracies and Running Times with MNIST dataset

PCs % Variance Running time Accuracy

3 23.1% 29.1 s 0.4574

5 33.3% 55.0 s 0.7179

10 48.9% 98.4 s 0.9251

15 58.0 % 172.5 s 0.9577

25 69.3% 282.6 s 0.9742

50 82.6 % 587.1 s 0.9761

100 91.5 % 1183.6 s 0.9738

200 96.7 % 2267.4 s 0.9720