View
238
Download
0
Category
Preview:
Citation preview
Multiway Generalized Canonical CorrelationAnalysis (MGCCA)
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3
1 Laboratoire des Signaux et Systemes (L2S, UMR CNRS 8506) CNRS - CentraleSupelec- Universite Paris-Sud - Gif-sur-Yvette
2 NeuroSpin/UNATI - CEA, Universite Paris-Saclay, Universite Paris-Saclay -France3 Brain and Spine Institute, Bioinformatics and Biostatistics platform - Paris
janvier
2/23
Motivation : Raman DataDesign
1 13 volunteers.
2 2 arms : moisturizer/placebo.
3 Raman spectroscopy on each arm.
Goal
Evaluate the efficiency of a moisturizer and identify differences in spectrum betweentreated/placebo.
Data
5 three-way tensors acquired at 0, 2, 4, 8 and 12 weeks.
2/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
3/23
Motivation : Raman Data
3/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
4/23
Introduction MGCCA Results Conclusion
Outline
1 From PLS to RGCCA
2 Multiway GCCA
3 Results on Raman data
4 Conclusion andPerspectives
4/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
5/23
Introduction MGCCA Results Conclusion
Notations
For matrices
1 X1, . . . ,XL are L data matrices.
2 Xl ∈ RI×Jl : a block.
3 wl ∈ RJl : a block-weight vector.
4 ξl = Xlwl ∈ RI : a block component.
5/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
6/23
Introduction MGCCA Results Conclusion
PLS/CCA/R-CCA
Canonical Correlation Analysis(CCA)
maxw1,w2
Var(X1w2)=1Var(X2w2)=1
Cor2 (X1w1, X2w2)
Partial Least Squares (PLS)
maxw1,w2
‖w1‖=‖w2‖=1
Cov2 (X1w1, X2w2)
Regularized-CCA (R-CCA)
maxw1,w2
Cov2 (X1w1, X2w2)
s.t. (1− τl)Var (Xlwl) + τl‖wl‖2 = 1, l = 1, 2.
6/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
6/23
Introduction MGCCA Results Conclusion
PLS/CCA/R-CCACanonical Correlation Analysis(CCA)
maxw1,w2
Var(X1w2)=1Var(X2w2)=1
Cor2 (X1w1, X2w2)
Partial Least Squares (PLS)
maxw1,w2
‖w1‖=‖w2‖=1
Cov2 (X1w1, X2w2)
Regularized-CCA (R-CCA)
maxw1,w2
Cov2 (X1w1, X2w2)
s.t. (1− τl)Var (Xlwl) + τl‖wl‖2 = 1, l = 1, 2.
6/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
6/23
Introduction MGCCA Results Conclusion
PLS/CCA/R-CCACanonical Correlation Analysis(CCA)
maxw1,w2
Var(X1w2)=1Var(X2w2)=1
Cor2 (X1w1, X2w2)
Partial Least Squares (PLS)
maxw1,w2
‖w1‖=‖w2‖=1
Cov2 (X1w1, X2w2)
Regularized-CCA (R-CCA)
maxw1,w2
Cov2 (X1w1, X2w2)
s.t. (1− τl)Var (Xlwl) + τl‖wl‖2 = 1, l = 1, 2.
6/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
7/23
Introduction MGCCA Results Conclusion
PLS/CCA/R-CCA with a figure
1 X ∼ N(
(0, 0) ,
(1 0.5
0.5 1
)).
2 y = 0 if x1 < 0, 1 otherwise.
7/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
7/23
Introduction MGCCA Results Conclusion
PLS/CCA/R-CCA with a figure
1 X ∼ N(
(0, 0) ,
(1 0.5
0.5 1
)).
2 y = 0 if x1 < 0, 1 otherwise.
RCCA
7/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
8/23
Introduction MGCCA Results Conclusion
Regularized Generalized Canonical CorrelationAnalysis (RGCCA) : Scheme
8/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
9/23
Introduction MGCCA Results Conclusion
RGCCA : Optimization problem
Optimization problem :
maxw1,...,wL
L∑k,l=1
ckl g (Cov (Xkwk ,Xlwl))
s.t. (1− τl)Var (Xlwl) + τl‖wl‖2 = 1, l = 1, . . . , L.
with g a continuous, convex and derivable function.
9/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
9/23
Introduction MGCCA Results Conclusion
RGCCA : Optimization problem
Optimization problem :
maxw1,...,wL
L∑k,l=1
ckl g (Cov (Xkwk ,Xlwl))
s.t. (1− τl)Var (Xlwl) + τl‖wl‖2 = 1, l = 1, . . . , L.
with g a continuous, convex and derivable function.
9/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
9/23
Introduction MGCCA Results Conclusion
RGCCA : Optimization problem
Optimization problem :
maxw1,...,wL
L∑k,l=1
ckl g (Cov (Xkwk ,Xlwl))
s.t. (1− τl)Var (Xlwl) + τl‖wl‖2 = 1, l = 1, . . . , L.
with g a continuous, convex and derivable function.
9/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
10/23
Introduction MGCCA Results Conclusion
Content
1 From PLS to RGCCA
2 Multiway GCCA
3 Results on Raman data
4 Conclusion andPerspectives
10/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
11/23
Introduction MGCCA Results Conclusion
Multiway Generalized Canonical Correlation Analysis(MGCCA)
Χ ..1l
I Χ ..2l
Jl
Χ .. Kl
l
JlJl
Jl
I
K l
I
K 1
Χ 1
Χ l
Χ l
J1 J L
I
KL
ΧL
ξ1 ξL
ξ l
Lateral slice
Frontal slice
Unfolding
c1 L
cl Lc1 l
Χ . j .1
Χ ..kl
11/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
12/23
Introduction MGCCA Results Conclusion
Kronecker product
a⊗ b =
[a1
a2
]⊗
b1
b2
b3
b4
=
a1
b1
b2
b3
b4
a2
b1
b2
b3
b4
=
a1b1
a1b2
a1b3
a1b4
a2b1
a2b2
a2b3
a2b4
From a vector of length 2 and a vector of length 4, a vector of
length 8 was created → estimation of 8 vs. 6 coefficients.
12/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
13/23
Introduction MGCCA Results Conclusion
Component via Kronecker product
ξl = Xl(wKl ⊗wJ
l ) = Xl(wKl ⊗ IJl )wJ
l
=
( Kl∑k=1
wKlk Xl
..k
)wJ
l
ξl = Xl(wKl ⊗wJ
l ) = Xl(IKl⊗wJ
l )wKl
=
( Jl∑j=1
wJlj X
l.j .
)wK
l
13/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
13/23
Introduction MGCCA Results Conclusion
Component via Kronecker product
ξl = Xl(wKl ⊗wJ
l ) = Xl(wKl ⊗ IJl )wJ
l
=
( Kl∑k=1
wKlk Xl
..k
)wJ
l
ξl = Xl(wKl ⊗wJ
l ) = Xl(IKl⊗wJ
l )wKl
=
( Jl∑j=1
wJlj X
l.j .
)wK
l
13/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
14/23
Introduction MGCCA Results Conclusion
MGCCA : Optimization problem
maxw1,...,wL
L∑k,l=1
ckl g (Cov (Xkwk ,Xlwl))
s.t. (1− τl)Var (Xlwl) + τl‖wl‖2 = 1
and wl = wKl ⊗wJ
l , l = 1, . . . , L
Optimized with a Block Coordinate Ascent approach with at eachupdate the following problem to solve(
vK?
l , vJ?
l
)= argmax
vKl ,vJl
‖vKl ⊗vJl ‖=1
vKl>
QlvJl → SVD of Ql of dimension Jl×Kl .
14/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
15/23
Introduction MGCCA Results Conclusion
Discussion
Advantages
1 Simple algorithm that monotonically converges toward astationary point.
2 The 3-way structure is taken into account.
3 Less weights to estimate : from Jl × Kl to Jl + Kl for eachblock.
15/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
16/23
Introduction MGCCA Results Conclusion
Content
1 From PLS to RGCCA
2 Multiway GCCA
3 Results on Raman data
4 Conclusion andPerspectives
16/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
17/23
Introduction MGCCA Results Conclusion
Motivation : Raman DataDesign
1 13 volunteers.
2 2 arms : moisturizer/placebo.
3 Raman spectroscopy on each arm.
Goal
Evaluate the efficiency of a moisturizer and identify differences in spectrum betweentreated/placebo.
Data
5 three-way tensors acquired at 0, 2, 4, 8 and 12 weeks.
17/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
18/23
Introduction MGCCA Results Conclusion
Raman MGCCA scheme
To adjust τl , l = 1, . . . , 5
1 10-folds MCCV.
2 LDA on concatenation ofblock components.
3 Itrain = 8/Itest = 5.
Results : In accuracy, 0.72(mean), 0.8 (median), 0.17(std).
18/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
19/23
Introduction MGCCA Results Conclusion
Raman MGCCA weights
wJl , l = 1, . . . , 5
wKl , l = 1, . . . , 5
19/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
20/23
Introduction MGCCA Results Conclusion
Content
1 From PLS to RGCCA
2 Multiway GCCA
3 Results on Raman data
4 Conclusion andPerspectives
20/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
21/23
Introduction MGCCA Results Conclusion
Conclusion
1 Simple algorithm that monotonically converges toward astationary point.
2 The 3-way structure is taken into account.
3 Less weights to estimate : from Jl × Kl to Jl + Kl .
4 Gain in interpretability thanks to vector weights specific toeach dimension.
21/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
22/23
Introduction MGCCA Results Conclusion
Perspectives
Future work :
Keep 3-way structure for higher components computation.
Apply MGCCA on more data sets : ADNI.
Develop Sparse MGCCA :
maxw1,...,wL
L∑k,l=1
ckl g(
w>k X>k Xlwl
)s.t. w>l Mlwl = 1 and wl = wK
l ⊗wJl , l = 1, . . . , L
‖wKl ‖1 ≤ cKl and/or ‖wJ
l ‖1 ≤ cJl
22/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
23/23
Introduction MGCCA Results Conclusion
Thank you for your attention !
Q U E S T I O N S ?
23/23
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
1/9
NotationsFor 3-way tensors
1 X1, . . . ,XL are L 3-way tensors.2 Xl ∈ RI×Jl×Kl : a block.3 Xl
..k ∈ RI×Jl : the kth frontal slice of Xl .4 Xl
.j . ∈ RI×Kl : the j th lateral slice of Xl .5 Xl = [Xl
..1, . . . ,Xl..kl, . . . ,Xl
..Kl] ∈ RI×JlKl : the matricized
version of Xl .
1/9
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
2/9
MGCCA : Change of variable
Idea : get rid of Ml . For that uses
Pl = I−1/2XlM−1/2l
vl = M1/2l wl
Ml = MKl ⊗MJ
l
vl = M1/2l wl = (MK
l )1/2wKl ⊗ (MJ
l )1/2wJl = vKl ⊗ vJl
Thus, the new optimization problem can be written as
maxv1,...,vL
f (v1, . . . vL) =L∑
k,l=1
clk g(
v>k P>k Plvl)
s.t. v>l vl = 1 and vl = vKl ⊗ vJl , l = 1, . . . , L (1)
2/9
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
3/9
MGCCA : Idea for the Algorithm (1)We want to find an update vl such thatf (v) ≤ f (v1, ..., vl−1, vl , vl+1, ..., vL). f is a continuouslydifferentiable multi-convex function, thus noting v = (v1, . . . , vL)
f (v1, ..., vl−1, vl , vl+1, . . . , vL) ≥ f (v)+∇l f (v)>(vl−vl) = `l(vl , v)
The solution that maximizes this minorizing function over vl isobtained by considering the following optimization problem :
v?l = rl(v) = argmaxvl=vKl ⊗vJl‖vKl ⊗vJl ‖=1
`l(vl , v)
In the end
f (v) = `l(vl , v) ≤ `l(v?l , v) ≤ f (v1, ..., vl−1, v?l , vl+1, ..., vL).
3/9
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
4/9
MGCCA : Idea for the Algorithm (2)It is possible to show that
∇l f (v1, ..., vL) = P>l
(L∑
k=1
clkg′(vl>P>l Pkvk)Pkvk
)= P>l zl .
Thus, introducing Ql = [(Pl..1)>zl , . . . , (Pl
..Kl)>zl ]
>, then(vK
?
l , vJ?
l
)= argmax
vKl ,vJl
‖vKl ⊗vJl ‖=1
zl>Pl(vKl ⊗ vJl )
= argmaxvKl ,v
Jl
‖vKl ⊗vJl ‖=1
vKl>
QlvJl
At the heart of MGCCA’s algorithm lies a Singular ValueDecomposition (SVD).
4/9
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
5/9
MGCCA : Algorithm
Algorithm 1 RGCCA algorithm for three-way data analysis1: Data : Xl , τl , g , ε2: Result : v1, . . . vl3: Initialization : choose random unit norm v0
l , l = 1, . . . , L ;4: s = 0 ;5: repeat6: for l = 1 to L do7:
zsl =
l−1∑k=1
clkg′(vsl>P>l Pkvs+1
k )P>l Pkvs+1k +
L∑k=l
clkg′(vsl>P>l Pkvsk )P>l Pkvsk
8: (vK
?
l , vJ?
l
)= argmax
vKl ,vJl‖vKl ⊗vJl ‖=1
vKl>
Qsl vJl
9: end for10: s = s + 1 ;11: until f (vs+1
1 , . . . , vs+1L )− f (vs1, . . . , v
sL) < ε
5/9
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
6/9
Discussion : Next components
Deflation is used. Same optimization problem is solved butreplacing each Xl , l = 1, . . . , L by
X(1)l = Xl − ξ
(1)l
((ξ
(1)l )>ξ
(1)l
)−1(ξ
(1)l )>Xl
6/9
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
7/9
Discussion : Choice of Ml
Ml = (1− τ)1
IX>l Xl + τ I
Same as in RGCCA. However, MJl and MK
l can appears
w>l Mlwl =
(wK
l ⊗wJl
)>[
(1− τ)1
IX>
l Xl + τ I
] (wK
l ⊗wJl
)= wK
l
> (IKl ⊗wJ
l
)>[
(1− τ)1
IX>
l Xl + τ I
] (IKl ⊗wJ
l
)wK
l
= wKl
>
(1− τ)1
I
( Jl∑j=1
w Jlj X
l.j.
)>( Jl∑j=1
w Jlj X
l.j.
)+ τ‖wJ
l ‖2IKl
wKl
= wKl
>MK
l wKl
7/9
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
8/9
Raman Data
8/9
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
9/9
Raman results
Table – 10 folds MCCV with ntrain = 16/ntest = 10 pairwised
Method Mean Median Std
Train
MGCCA hierarchical 1 1 0
MGCCA complete 1 1 0
Parafac 0.95 1 0.11
RGCCA hierarchical 1 1 0
Test
MGCCA hierarchical 0.72 0.8 0.17
MGCCA complete 0.74 0.8 0.16
Parafac 0.70 0.8 0.19
RGCCA hierarchical 0.70 0.8 0.199/9
A. Gloaguen 1,2, C. Philippe2, V.Frouin2, L. Le Brusquet1 & A. Tenenhaus 1,3 L2S / NeuroSpin Multiway Generalized Canonical Correlation Analysis (MGCCA)
Recommended