Embed Size (px)
Citation preview
PCA Extension
By Jonash
Outline
• Robust PCA
• Generalized PCA– Clustering points on a line– Clustering lines on a plane– Clustering hyperplanes in a space
Robust PCA
• Rrbust Principal Component Analysis for Computer Vision– Fernando De la Torre– Mochael J. Black
• CS, Brown University
PCA is Least-Square Fit
PCA is Least-Square Fit
Robust Statistics
• Recover the best fit for the majority of the data
• Detect and reject outliers
Robust PCA
Robust PCA
Robust PCA
• Training images
Robust PCA
• Naïve PCA
• Simply reject
• Robust PCA
RPCA
• In traditional PCA, we minimize• Σn
i = 0 (di – B BT di)2 = Σni = 0 (di - Bci)2
• EM PCA Limσ -> 0(D = BC + σ2I)
E-step C = (BTB)-1BTDM-step B = DCT(CCT)-1
di B
BTdi = ci
BBTdi
RPCA
• Xu and Yuille [1995] tries to minimize
– Σni = 1 [ V i(di – B ci)2 + n(1-Vi) ]
– Hard to solve (continuous + discrete)
RPCA
• Gabriel and Zamir [1979] tries to minimize
– Σni = 1 Σd
p = 1 [ wpi(dpi – Bci)2]
– Impratical for high dimension
– “Low rank approximation of matrices by least squares with any choice of weights” 1979
RPCA
• Idea is to use a robust function ρ– Geman-McClureρ (x,σ) = x2/(x2 + σ2)
– Σni = 1 Σd
p = 1 ρ[ (dpi–μp –Σkj = 1 bpjcji), σp]
– Approximated by local quadratic function– Use gradient descent– The rest is nothing but heuristics
RPCA
Robust PCA - Experiment
• 256 training images (120x160)
• Obtain 20 RPCA basis
• 3 hrs on 900MHz Pentinum III in Matlab
Outline
• Robust PCA
• Generalized PCA– Clustering points on a line– Clustering lines on a plane– Clustering hyperplanes in a space
Generalized PCA
• Generalized Principal Component Analysis– Rene Vidal– Yi Ma– Shankar Sastry
• UC Berkley and UIUC
GPCA
GPCA Example 1
GPCA Example 2
GPCA Example 3
GPCA Goals
1. # of subspaces and their dimension2. Basis for subspace3. Segmentation of data
GPCA Ideas
• Union of subspaces = certain polynomials
Outline
• Robust PCA
• Generalized PCA– Clustering points on a line– Clustering lines on a plane– Clustering hyperplanes in a space
GPCA 1D Case
GPCA 1D Case Cont’d
GPCA 1D Case Cont’d
MN = n+1
To have a unique solution, rank(Vn) = n = Mn- 1
unknowns
GPCA 1D Example
• n = 2 groups• pn(x) = ( x – μ1) ( x – μ2)
• No polynomial of degree 1• Infinite polynomial of degree 3
• pn(x) = x2 + c1x + c2 => Polynomial factor
Outline
• Robust PCA
• Generalized PCA– Clustering points on a line– Clustering lines on a plane– Clustering hyperplanes in a space
GPCA 2D Case
• L j = { X = [x, y]T: bj1x + bj2y = 0 }
• (b11x + b12y = 0) or (b21x + b22y = 0)…
GPCA 2D Case Cont’d
• (b11x + b12y = 0) or (b21x + b22y = 0)…
• Pn(x) = (b11x + b12y)…(bn1x + bn2y) = 0 = Σck xn-k yk
GPCA 2D Case Cont’d
• Take n = 2 for example…• p2(x) = (b11x + b12y)(b21x + b22y)
• ▽p2(x) = (b21x + b22y)b1 + (b11x + b12y)b2
, bj = [bj1, bj2] T
• if x ~ L1, then ▽p2(x) ~ b1, otherwise ~ b2
GPCA 2D Case Cont’d
• Given that {yj ε Lj}, the normal vector of Lj is bj ~ ▽pn(yj)
• 3 things…1. Determine “ n ” as min{ j: rank(Vj) = j }
2. Solve cn for Vncn = 03. Find normal vector bj
Outline
• Robust PCA
• Generalized PCA– Clustering points on a line– Clustering lines on a plane– Clustering hyperplanes in a space
GPCA Hyperplanes
• Still assume d1 = … = dn = d = D – 1• Sj = { bj
Tx = bj1x1 + bj2x2 + … + bjDxD = 0}
GPCA Hyperplanes
MN = C(D+n-1, D)
GPCA Hyperplanes
GPCA Hyperplanes
• Since we know n, we can solve for ck
• ck => bk by ▽pn(x)
• If we know yj on each Sj, finding bj will be easy
GPCA Hyperplanes
• One point yj on each hyperplane Sj
• Consider a random line L = t * v + x0
• Obtain yj by intersecting L and Sj
– yj = tj * v + x0
– Find roots tj by … Pn(t v + xo)
GPCA Hyperplanes
• Summarize1. We want to find n to solve for c
2. To get b (normal) for each S, find ▽pn(x) 3. To get label j, solve pn(yj = tj * v + x0) = 0
One More Thing
One More Thing
• Previously we assume d1 = … =dn= D – 1• Actually we cannot assume that…
• Please read section 4.2 & 4.3 … by yourself– Discuss how to recursively reduce dimensi
on
