ERPWAVELAB 1st International Summer School in Biomedical Engineering1st International Summer School in Biomedical Engineering August 8, 2006 1st International

Morten MørupTechnical University of Denmark

[email protected]

ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8, 2006 August 8, 2006

Analyzing the Wavelet Transformed EEG using Non-negative Matrix and Tensor Factorization

An introduction to ERPWAVELAB

ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8 2006 August 8 2006


[email protected]

Parts of the work done in collaboration with

Lars Kai Hansen, ProfessorDepartment of Signal Processing

Informatics and Mathematical Modeling,Technical University of Denmark

Sidse M. Arnfred, Dr. Med. PhDCognitive Research Unit

Hvidovre HospitalUniversity Hospital of Copenhagen


[email protected]

ERPWAVELAB 1st International Summer School in Biomedical Engineering August 8, 2006 August 8, 2006

•The continuous wavelet transform and measures of the event related ERP in the time-frequency domain•Introduction to NMF and extensions to tensor decompositions (PARAFAC & TUCKER)•Accessing significance•A demonstration of ERPWAVELAB•Discussion

OUTLINE



[email protected]

time

time

frequency

Continuous Wavelet transform

Complex Morlet wavelet - Real part - Complex part

Absolute value of wavelet coefficient

Captures frequency changes through time

ie



[email protected]

Continuous Wavelet transform (continued)

epoc

h

chan

nel

time-frequency

epoc

h

chan

nel

time

epochtimechannel X epochfrequencytimechannel X



[email protected]

The Vector strength

Vectors coherent, i.e. correlated Vectors incoherent, i.e. uncorrelated

Vector strength a measure of coherence



[email protected]

Measures of the event related ERP in the time-frequency domain

ERSP

WTav

ITPC

avWT



[email protected]

Measures of the event related ERP in the time-frequency domain (cont.)

Since scalp works as low pass filter it is customary to normalize X before calculating each measure. Frequently used normalizations are:(where tb are points in the baseline region and Tb the total number of baseline samples)

fntfcX

ntfcXT

ntfcXb

b

T

tb

b

),,,(or

),,,(1

),,,(



[email protected]

ERPWAVELAB demonstration, tutorial dataset 1

ERSP

WTav avWT INDUCED

ITPC



[email protected]

f’=f, t’=t



[email protected]

Multi-channel decomposition

time-frequency activities appear to be similar across channels but varying in strength.

Motivates to decompose the activity into similar time-frequency signatures varying in strength in the recording channels.Thus, this form of decomposition is primarily useful for data exploratory purposes giving very easy summaries of what types of activities are present in the data. Only when the measures of interest can be assumed linear and no cancellation between sources are present the decomposition can also reveal the underlying true sources.

≈



[email protected]

Factor Analysis

dW

dH

Spearman ~1900

VWH

d

Vtests x subjects Wtests x intelligencesHintelligencesxsubject

test

s

Subjects Subjects

test

s

Int.

Int.

Non-negative Matrix Factorization (NMF): VWH s.t. Wi,d,Hd,j0

(~1970 Lawson, ~1995 Paatero, ~2000 Lee & Seung)



[email protected]

Non-negative matrix factorization (NMF)NMF: VWH s.t. Wi,d,Hd,j0

Multiplicative updates: Let C be a given cost function

Positive termNegative term



[email protected]

Non-negative matrix factorization (NMF)

(Lee & Seung - 2001)

NMF gives Part based representation (Lee & Seung – Nature 1999)



[email protected]

The NMF decomposition is not unique

Simplical Cone

NMF only unique when data adequately spans the positive orthant (Donoho & Stodden - 2004)

HWH)(WP)(PWHV -1 ~~

z

y

x

Convex Hull

z

y

x

Positive Orthant

z

yx

Simplical Cones



[email protected]

Sparse Coding NMF (SNMF)

(Eggert & Körner, 2004) (Mørup & Schmidt, 2006)



[email protected]

Why sparseness?

• Ensures uniqueness• Eases interpretability

(sparse representation factor effects pertain to fewer dimensions)

• Can work as model selection(Sparseness can turn off excess factors by letting them become zero)

• Resolves over complete representations (when model has many more free variables than data points)



[email protected]

time-frequency Subjec

ts/Con

dition

/Tria

ls

chan

nel

Often extra modalities such as subjects, conditions and trials are present, consequently the data forms a tensor.

Need for tensor decomposition



[email protected]

Higher Order Non-negative Matrix Factorization

2dA

1dA

dW

dH

1dA

3dA

1A

3A

2A

C

Factor Analysis PARAFAC TUCKER

D

ddidiii

12121

HWV 3

1

21

321321 di

D

ddidiiii AAA

V 3

3

2

2

33

1

1

2211321321

321J

j

J

jji

J

jjijijjjiii AAACV

= 321

13

1

321

1211111321 ji

J

jjjjijijjjiii AAA

CV



[email protected]

Uniqueness• Although PARAFAC in general is unique under mild conditions, the

proof of uniqueness by Kruskal is based on k-rank*. However, the k-rank does not apply for non-negativity**.

• TUCKER model is not unique, thus no guaranty of uniqueness.Imposing sparseness useful in order to achieve unique decompositions

Tensor decompositions known to have problems with degeneracy, however when imposing non-negativity degenerate solutions can’t occur***

*) k-rank: The maximum number of columns chosen by random of a matrix certain to be linearly independent. **) L.-H. Lim and G.H. Golub, 2006.***) See L.-H. Lim - http://www.etis.ensea.fr/~wtda/Articles/wtda-nnparafac-slides.pdf



[email protected]

Example why Non-negative PARAFAC isn’t unique

10

002

½1

½12,

10

10½

½1

½1:

10

002

½1

½12,

10

00½

00

10½

½1

½1:

10

003

00

10

01

012,

10

00

00

10

01

01:

20

00

01

01

11

11,

11

11:

:3

22:

32

12

11

01)

1

2(

11

11

11

11

11

01)

0

1(

11

11

12

01,

11

01,

11

11

21

21

21

21

2

1

)3()2()1(

XX

XX

XX

XX

X

X

AAA

IV

III

II

I

ranknegativeNon

satisfiedFKKKconditionKruskal

diag

diag

CBA

T

T



[email protected]

PARAFAC model estimation

2dA

1dA

3dA

3

1

21

321321 di

D

ddidiiii AAA

V

V

T123

33

AAZ

ZAV 3

T231

111

AAZ

ZAV

T132

222

AAZ

ZAV

Thus, the PARAFAC model is by the matricizing operationThus, the PARAFAC model is by the matricizing operationestimated straight forward from regular NMF estimationestimated straight forward from regular NMF estimation

JJ BABABABA 2211



[email protected]

1dA

1A

3A

2A

TUCKER

3

3

2

2

33

1

1

2211321321

321J

j

J

jji

J

jjijijjjiii AAACV

TUCKER model estimation

T123

3

33

AACZ

ZAV 3

T23)1(

1

111

AACZ

ZAV

T132

2

222

AACZ

ZAV

)( 123 AAA CV vecvec



[email protected]

Algorithm outline (TUCKER) (PARAFAC follows by setting C=I)



[email protected]



[email protected]

Accessing Significance

• Comparison to known distribution • Bootstrapping• Cross validation



[email protected]

Comparison to known distribution

(Mardia, Directional Statistics)

Rayleigh distributedRed: Theoretical mean value of N-½

Black: Mean value estimated by bootstrapping

Normal distributed

Random ITPC and ERPCOH corresponds to a random walk in the complex planethus is Rayleigh distributed.

2:

1:

)(:

2

2

2

2

2

22

mean

ecdf

ex

xfpdf

x

x



[email protected]

Bootstrapping

1) Randomly select Data from the epochs to form new datasets (each epoch might be represented 0, 1 or several times in the datasets). 2) Calculate the measure of interest for each of these datasets.3) Evaluate the values found to the distribution of values found by the bootstrap datasets.



[email protected]

Cross validation

• Split dataset into exploratory and confirmatory datasets.

• Find significant activity in exploratory dataset

• See if this activity is also significant in confirmatory dataset

Correcting for multiple comparison by bootstrapping very expensive and often too conservative. Thus Cross validation useful.



[email protected]

– Dataset generation– Single subject analysis

Artifact rejection in the time frequency domainNMF decompositionCross coherence tracking

– Multi subject analysisClusteringAnalysis of Variance (ANOVA)Tensor decomposition

ERPWAVELAB Tutorial:

The toolbox is free to download from www.erpwavelab.com



[email protected]

Epilog: Some History of PARAFAC and EEG

• Harshman (1970) (Suggested its use on EEG)• Möcks (1988) (Topographic Component Analysis)

ERP of (channel x time x subject)• Field and Graupe (1991)

ERP of (channel x time x subject)• Miwakeichi et al. (2004)

EEG of (channel x time x frequency)• Mørup et al. (2005)

ERP of ITPC (channel x time x frequency x subject x condition)



[email protected]

References

Bro, R., 1998. Multi-way Analysis in the Food Industry: Models, algorithms and Applications. Amsterdam, Copenhagen.Bro, R.,Jong, S. D., 1997. A fast non-negativity-constrained least squares algorithm. J. Chemom. 11, 393–401Carroll, J. D. and Chang, J. J. Analysis of individual differences in multidimensional scaling via an N-way generalization of "Eckart-Young" decomposition, Psychometrika 35 1970 283—319Delorme, A.,Makeig, S., 2004. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods 134, 9-21Donoho, D. and Stodden, V. When does non-negative matrix factorization give a correct decomposition into parts? NIPS2003Eggert, J. and Korner, E. Sparse coding and NMF. In Neural Networks volume 4, pages 2529-2533, 2004Field, Aaron S.; Graupe, Daniel “Topographich Component (Parallel Factor) analysis of Multichannel Evoked Potentials: Practical Issues in Trilinear Spatiotemporal Decomposition” Brain Topographa, Vol. 3, Nr. 4, 1991 Fiitzgerald, D. et al. Non-negative tensor factorization for sound source separation. In proceedings of Irish Signals and Systems Conference, 2005Kruskal, J.B. Three-way analysis: rank and uniqueness of trilinear decompostions, with application to arithmetic complexity and statistics. Linear Algebra Appl., 18: 95-138, 1977Harshman, R. A. Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-modal factor analysis},UCLA Working Papers in Phonetics 16 1970 1—84Herrmann, C. S., Grigutsch, M.,Busch, N. A., 2005. EEG oscillations and wavelet analysis.Lathauwer, Lieven De and Moor, Bart De and Vandewalle, Joos MULTILINEAR SINGULAR VALUE DECOMPOSITION.SIAM J. MATRIX ANAL. APPL.2000 (21)1253–1278Lee, D.D. and Seung, H.S. Algorithms for non-negative matrix factorization. In NIPS, pages 556-462, 2000Lee, D.D and Seung, H.S. Learning the parts of objects by non-negative matrix factorization, NATURE 1999Lim, Lek-Heng - http://www.etis.ensea.fr/~wtda/Articles/wtda-nnparafac-slides.pdf

Lim, L.-H. and Golub, G.H., "Nonnegative decomposition and approximation of nonnegative matrices and tensors," SCCM Technical Report, 06-01, forthcoming, 2006.Mardia, K. V.,Jupp, P. E., 1999. Directional Statistics. WILEY & SONS 76-77Miwakeichi, F., Martinez-Montes, E., Valdes-Sosa, P. A., Nishiyama, N., Mizuhara, H., Yamaguchi, Y., 2004. Decomposing EEG data into space-time-frequency components using Parallel Factor Analysis. Neuroimage 22, 1035-1045.

Möcks, J., 1988. Decomposing event-related potentials: a new topographic components model. Biol. Psychol. 26, 199-215.

Mørup, M and Hansen, L.K and Herman, C.S. and Parnas, Josef and Arnfrede, Sidse M. “Parallel Factor Analysis as an exploratory tool for wavelet transformed event –related EEG” NeuroImage 20, 938-947 (2006)

Mørup, M. and Hansen, L.K.and Arnfred, S.M.Decomposing the time-frequency representation of EEG using nonnegative matrix and multi-way factorization Technical report, Institute for Mathematical Modeling, Technical University of Denmark, 2006a

Mørup, M. and Schmidt, M.N. Sparse non-negative matrix factor 2-D deconvolution. Technical report, Institute for Mathematical Modeling, Technical University of Denmark, 2006b

Mørup, M. and Hansen, L.K.and Arnfred, S.M. Algorithms for Sparse Higher Order Non-negative Matrix Factorization (HONMF), Technical report, Institute for Mathematical Modeling, Technical University of Denmark, 2006e

Tamara G. Kolda Multilinear operators for higher-order decompositions technical report Sandia national laboratory 2006 SAND2006-2081.

Tucker, L. R. Some mathematical notes on three-mode factor analysis Psychometrika 31 1966 279—311

Welling, M. and Weber, M. Positive tensor factorization. Pattern Recogn. Lett. 2001

Documents

ERPWAVELAB 1st International Summer School in Biomedical Engineering1st International Summer School in Biomedical Engineering August 8, 2006 1st International