Lecture09_ICAIntro

5/14/2018 Lecture09_ICAIntro - slidepdf.com

http://slidepdf.com/reader/full/lecture09icaintro 1/9

IntroductionIntroduction to Independent Component Analysisto Independent Component Analysis

Vince D. Calhoun, Ph.D.Vince D. Calhoun, Ph.D.Chief Technology Officer &Chief Technology Officer &

Director, Image Analysis & MR Research Director, Image Analysis & MR ResearchThe Mind Research Network The Mind Research Network

Associate Professor, Electrical and Computer Engineering, Associate Professor, Electrical and Computer Engineering, Neurosciences & Computer Science Neurosciences & Computer Science

The University of New MexicoThe University of New Mexico

OverviewOverview

•• (Brief) ICA Intro(Brief) ICA Intro

•• ICA of fMRIICA of fMRI

•• Sorting/CalibrationSorting/Calibration

•• ValidationValidation

•• Demo singleDemo single--subject ICAsubject ICA

fMRI fMRI CourseCourse 2X A S

Modeling the Brain?Modeling the Brain?

Results

Modeling Discussion

From “Science with a Smile”

by Subramanian Raman

fMRI fMRI CourseCourse 3

•• “All models are wrong, but some are useful!”“All models are wrong, but some are useful!”

•• “All models are wrong.” G.E. Box (1976) quoted by Marks Nester in, “An applied statistician’s“All models are wrong.” G.E. Box (1976) quoted by Marks Nester in, “An applied statistician’s

creed,” Applied Statistics, 45(4):401creed,” Applied Statistics, 45(4):401--410, 1996.410, 1996.

•• “I believe in ignorance“I believe in ignorance--based methods because humans have a lot of ignorance and we based methods because humans have a lot of ignorance and we

should play to our strong suit.”should play to our strong suit.”

•• Eric Lander, Whitehead Institute, M.I.T.Eric Lander, Whitehead Institute, M.I.T.

Mixing matrix A

Observations

Blind Source Separation: The Cocktail Party ProblemBlind Source Separation: The Cocktail Party Problem


Sources

Independent Component AnalysisIndependent Component Analysis• Goal:

Separate sources from a linear mixture

• Model:

X=AS

X: Mixture

A: Mixing coefficients


•• SS== W XW X

•• WW== AA−−11

• Assumptions• Linear mixing

• Independence of sources

• Non-gaussian sources

ICA vs PCAICA vs PCA 1 2 1 2Uncorrelated: E y y E y E y

1 2 1 2

1 2 1 2

Independent: ,

p y y p y p y

E h y h y E h y E h y


PCA finds directions of maximalPCA finds directions of maximalvariance (using second ordervariance (using second order

statistics)statistics)

ICA finds directions which maximizeICA finds directions which maximizeindependence (using higher order statistics)independence (using higher order statistics)



ICA vs. PCAICA vs. PCA


PCA finds directions of maximal variancePCA finds directions of maximal variance(using second order statistics)(using second order statistics)

ICA vs. PCAICA vs. PCA


ICA finds directions which maximize independenceICA finds directions which maximize independence(using higher order statistics)(using higher order statistics)

•• Mixing simple signals: sinus + chainsaw.Mixing simple signals: sinus + chainsaw.

ICA ExampleICA Example

fMRI fMRI CourseCourse 9From: Chap. 14.6, Friedman, Hastie, Tibshirani:From: Chap. 14.6, Friedman, Hastie, Tibshirani: The elements of statistical learningThe elements of statistical learning..

InfomaxInfomax Bell&Sejnowski ‘95 Bell&Sejnowski ‘95

MaximumMaximum

LikelihoodLikelihoodGaeta ’90, Pearlmutter ‘96 Gaeta ’90, Pearlmutter ‘96

MaximumMaximum

Lee ‘98 Lee ‘98

Cardoso ’96 Cardoso ’96

Pearlmutter ’97 Pearlmutter ’97 (if the nonlinearities in the NN (if the nonlinearities in the NN

are chosen as the cdf’s)are chosen as the cdf’s)

Cardoso ’99Cardoso ’99(represent likelihood by(represent likelihood by

K K--L distance between L distance between

observed and factorized observed and factorized

density)density)

Nonlinear PCA Nonlinear PCA

Karhunen ’97 Karhunen ’97

Girolami & Fyfe ‘97 Girolami & Fyfe ‘97


Comon ’94Comon ’94

BlindBlind

DeconvolutionDeconvolution Lambert ’96/Bussgangs Lambert ’96/Bussgangs

Lee ‘98 Lee ‘98

Min. Mutual Info.Min. Mutual Info.Comon ’94Comon ’94

Lee ’98 Lee ’98

Hyvarinen ‘99 Hyvarinen ‘99(if constrained to be uncorrelated)(if constrained to be uncorrelated)

Oja ’97,Oja ’97,

Karhunen & Jautsensalo ’94Karhunen & Jautsensalo ’94

ICA Maximizes NongaussianityICA Maximizes Nongaussianity

{ }Y E Y 2 2{ ( )}Y

E Y E Y 2

4 24 3Y E Y E Y 3

3 Y E Y

meanmean variancevariance skewnessskewness kurtosiskurtosis


•• Many realMany real--world data sets haveworld data sets havesupergaussiansupergaussiandistributionsdistributions

•• The random variables takeThe random variables takerelatively more often values thatrelatively more often values thatare very close to zero or areare very close to zero or arelargelarge

GaussianGaussian supergaussiansupergaussian

FastICA demo (mixtures)FastICA demo (mixtures)




FastICA demo (whitened)FastICA demo (whitened)


FastICA demo (step 1)FastICA demo (step 1)








FastICA demo (step 5FastICA demo (step 5 -- end)end)




ICA of FMRIICA of FMRI


General Linear Model

1. Model1. Model(1 or more(1 or more

Regressors)Regressors)

oror

2. Data2. Data

i x j

y j


RegressionRegressionResultsResults

3. Fitting the3. Fitting theModel to theModel to theData at eachData at eachvoxelvoxel

0

1

ˆ ˆ M

i i

i

y j x j e j

Voxels T i m e Data(X) = G

“Activation maps”Corresponding to columns of G

β̂

Time courses

Designmatrix

General Linear Model (GLM)General Linear Model (GLM)

The GLM is by farThe GLM is by farthe most commonthe most commonapproach toapproach toanalyzing fMRIanalyzing fMRIdata. To use thisdata. To use thisapproach, one needsapproach, one needsa model for the fMRIa model for the fMRItime coursetime course


Voxels T i m e Data(X) = Components (C)1ˆ W

Time courses

Spatially IndependentComponents

Mixing matrix

Independent Component Analysis (ICA)Independent Component Analysis (ICA)

In spatial ICA,In spatial ICA,there is no modelthere is no modelfor the fMRI timefor the fMRI timecourse, this iscourse, this isestimated alongestimated alongwith thewith thehemodynamichemodynamicsource locationssource locations

Voxels T i m e Data(X) = Components (C)

1ˆ W

Spatially IndependentComponents

Mixing matrix

A little more detailA little more detail

†R


Time courses

TC

{ {

ICA ExampleICA Example


ICA Halloween (Un)Mixer!ICA Halloween (Un)Mixer!

==

→→ TimeTime

backgroundbackground

candle 1candle 1

X = AX = A ×× SS


Candle outCandle out

candle 2candle 2

candle 3candle 3



ICA of fMRIICA of fMRI

++

fMRI data,fMRI data, xx

The ICA modelThe ICA modelassumes the fMRIassumes the fMRI

data,data, xx, is a linear, is a linearmixture ofmixture ofstatisticallystatistically

independentindependentsources,sources, ss..


A 1 2

T s ss

The goal of ICA is toThe goal of ICA is toseparate the sourcesseparate the sourcesgiven the mixed datagiven the mixed dataand thus determineand thus determinethethe ss andand AA matricesmatrices

1 2 1 2, p s s p s p s

x As

Source 1Source 1

Source 2Source 2

Time course 1Time course 1

Time course 2Time course 2

Time courses

GLM ICA

Input:

Data

Input:

Data


Output:

Activation Maps

Activation Maps(Components)

Time Courses

Output:

ICA of fMRI DataICA of fMRI Data


Signal TypesSignal Types

Task relatedTask related

CardiacCardiac


MotionMotion

Vasomotor oscillation/Vasomotor oscillation/

High order visualHigh order visual

Motion ArtifactMotion Artifact


Motion-related signal due to mouth movement from

inferior temporal and orbitofrontal regions

Hemodynamic ModelHemodynamic Model

Artifact Detection and ReductionArtifact Detection and Reduction

Eye movementsEye movements N/2 Nyquist Ghost N/2 Nyquist Ghost

Note: PREPROCESSING MAY DIFFER FOR Art. Hunting ApproachNote: PREPROCESSING MAY DIFFER FOR Art. Hunting Approach


Source: Christian Beckmann’s “Little Shop of fMRI Horrors”:Source: Christian Beckmann’s “Little Shop of fMRI Horrors”:http://www.fmrib.ox.ac.uk/~beckmann/homepage/academis/littleshop/http://www.fmrib.ox.ac.uk/~beckmann/homepage/academis/littleshop/



Spatial versus Temporal ICASpatial versus Temporal ICA

•• Does it matter?Does it matter?

•• Why is spatial ICA more common?Why is spatial ICA more common?

•• Some examples:Some examples:


SICA TICASPMTemporally and Spatially lowTemporally and Spatially low--correlated Componentscorrelated Components


Spatially Dependent Components

SICA TICASPM


A few pointsA few points

•• ICA is not modelICA is not model--free! (but does make nofree! (but does make no

assumptions about shape of time course)assumptions about shape of time course)

•• Fishing vs. HypothesizingFishing vs. Hypothesizing

•• ICA is a dataICA is a data--driven approach, but that does not meandriven approach, but that does not mean

you should automatically associate it with fishingyou should automatically associate it with fishing


ex eex e analys sanalys s vs. ypo es svs. ypo es s-- asease studystudy

•• The key here is what are the questions being asked, andThe key here is what are the questions being asked, and

what is the approach that will be used to answer thesewhat is the approach that will be used to answer these

questionsquestions

•• There are tools available for asking focusedThere are tools available for asking focused

questions about an ICA of fMRI analysisquestions about an ICA of fMRI analysis

Uses of ICAUses of ICA

•• Improving fit to task Improving fit to task--related componentsrelated components

•• Find areas of ‘activation’ which respond in a moreFind areas of ‘activation’ which respond in a more

complex way to an external stimuluscomplex way to an external stimulus

•• Artifact Reduction/FilteringArtifact Reduction/Filtering

•• Examination of temporally coherent, but notExamination of temporally coherent, but not


necessar y asnecessar y as --re a e componen sre a e componen s•• Data exploration of unpredicted structureData exploration of unpredicted structure

SortingSorting




Ambiguities of ICA: Sorting/ScalingAmbiguities of ICA: Sorting/Scaling

•• ICA is modeling the data as a linear combinationICA is modeling the data as a linear combination

of images and time coursesof images and time courses

•• Why is sorting necessary?Why is sorting necessary?•• Permutation ambiguity: X=AS=(APPermutation ambiguity: X=AS=(AP--11)(PS))(PS)

•• Why is scaling/calibration necessary?Why is scaling/calibration necessary?--


•• ca ng am gu y: = =ca ng am gu y: = = --

1 1 2 2* *data tc im tc im E

1 1 2 2

1 1( * ) * * ( * )* *data a tc im b tc im E

a b

Types of SortingTypes of Sorting

•• Temporal SortingTemporal Sorting

•• CorrelationCorrelation

•• Multiple regressionMultiple regression•• Others? (skew, kurtosis, power spectra, etc.)Others? (skew, kurtosis, power spectra, etc.)

•• Spatial SortingSpatial Sorting

••


•• Maximum value (w/i mask)Maximum value (w/i mask)

•• Multiple regressionMultiple regression

•• MultiMulti--variate sortingvariate sorting

•• SVM Approaches (Formisano)SVM Approaches (Formisano)

Ri ht

+

ExampleExample


0 90 180 270 360

Left

t (secs)

+

+


Number of Components Number of Components

•• Too manyToo many --> over > over--splitting of the componentssplitting of the components

•• Too fewToo few --> over > over--clumping of the componentsclumping of the components

•• How to choose?How to choose?

•• Between 20 and 40 appears to be a reasonable choiceBetween 20 and 40 appears to be a reasonable choice

for typical fMRI experimentfor typical fMRI experiment

••


and other ICA software programs (AIC/MDL/BIC)and other ICA software programs (AIC/MDL/BIC)

•• PostPost--ICA clustering is also used to address this issueICA clustering is also used to address this issue

Number of Components (Order Selection) Number of Components (Order Selection)

1ˆ2 2 1 12

N AIC N M K N NK N

L

1 1ˆ 1 1 ln2 2

N MDL N M K N NK N M

L

1

1

1

...ˆ ln1

...

K N N K

N

N K K N

L


M=number of voxels M=number of voxelsK=number of time pointsK=number of time pointsN=number of sourcesN=number of sources λ λ =eigenvalues from PCA=eigenvalues from PCA

Correction for correlated samples [Y. Li, T. Adali, and V. D. Calhoun, "Sample Dependence

Correction For Order Selection In FMRI Analysis," in Proc. ISBI , Washington, D.C., 2006.]

[V. D. Calhoun, T. Adali, G. D. Pearlson, and J. J. Pekar, "A Method for Making Group Inferences

From Functional MRI Data Using Independent Component Analysis," Hum. Brain Map. , vol. 14, pp.

140-151, 2001.]




ValidationValidation•• Algorithm Differences Algorithm Differences

•• Esposito F, Formisano E, Seifritz E, Goebel R, Morrone R, Tedeschi G,Esposito F, Formisano E, Seifritz E, Goebel R, Morrone R, Tedeschi G,Salle FD. 2002.Salle FD. 2002. Spatial Independent Component Analysis of FunctionalSpatial Independent Component Analysis of FunctionalMRI TimeMRI Time--Series: to What Extent Do Results Depend on the AlgorithmSeries: to What Extent Do Results Depend on the AlgorithmUsed? Hum Brain Map 16:149Used? Hum Brain Map 16:149--57.57.

•• Algorithm & Preprocessing Differences Algorithm & Preprocessing Differences•• Calhoun, V. D., Adali, T., and Pearlson, G. D. 2004. IndependentCalhoun, V. D., Adali, T., and Pearlson, G. D. 2004. Independent

Components Analysis Applied to fMRI Data: A Generative Model for Components Analysis Applied to fMRI Data: A Generative Model for Validating Results. Journal of VLSI Signal Proc Systems.Validating Results. Journal of VLSI Signal Proc Systems. vol. 37, pp.vol. 37, pp.281281--291.291.


•• uster a at onuster a at on•• Himberg J, Hyvarinen A, Esposito F. 2004. Validating the IndependentHimberg J, Hyvarinen A, Esposito F. 2004. Validating the Independent

Components of Neuroimaging Time Series Via Clustering andComponents of Neuroimaging Time Series Via Clustering andVisualization. Neuroimage 22(3):1214Visualization. Neuroimage 22(3):1214--22.22.

•• Test/retest PerformanceTest/retest Performance•• Nybakken GE, Quigley MA, Moritz CH, Cordes D, Haughton VM, Nybakken GE, Quigley MA, Moritz CH, Cordes D, Haughton VM,

Meyerand ME. 2002. TestMeyerand ME. 2002. Test--Retest Precision of Functional MagneticRetest Precision of Functional MagneticResonance Imaging Processed With Independent Component Analysis.Resonance Imaging Processed With Independent Component Analysis. Neuroradiology 44(5):403 Neuroradiology 44(5):403--6.6.

“Hybrid” fMRI Experiment“Hybrid” fMRI Experiment

Ground Truth


Mixed with fMRI Data

Impact of preprocessing/algorithms/etcImpact of preprocessing/algorithms/etc

( ) ln p

D p d p

s

s

u

ξs u ξ ξ

ξ

•• Define sourcesDefine sources

Criterion: KullbackCriterion: Kullback--Leibler (KL)Leibler (KL)divergencedivergence


•• enera e sourcesenera e sources

•• For all:For all:

•• Add noiseAdd noise

•• SmoothSmooth

•• Reduce (PCA, cluster, etc.)Reduce (PCA, cluster, etc.)

•• Unmix (Info., fastICA, jade, etc.)Unmix (Info., fastICA, jade, etc.)

•• Evaluate (KL)Evaluate (KL)

•• min(KL) is winner min(KL) is winner

V.D.Calhoun, T.Adali, and G.D.Pearlson, "Independent Components Analysis Applied to FMRI Data: AV.D.Calhoun, T.Adali, and G.D.Pearlson, "Independent Components Analysis Applied to FMRI Data: AGenerative Model for Validating Results," Journal of VLSI Signal Proc. Systems, 2004.Generative Model for Validating Results," Journal of VLSI Signal Proc. Systems, 2004.

Comparison of Different AlgorithmsComparison of Different Algorithms


N. Correa, T. Adali, and V. D. Calhoun "Performance of Blind Source Separation Algorithms for fMRI Analysis," Mag.Res.Imag., 2006 (submitted).

N. Correa, T. Adali, Y. Li, and V. D. Calhoun, "Comparison of Blind Source Separation Algorithms for FMRI Using a New Matlab Toolbox:

GIFT," in Proc. ICASSP, Philadelphia, PA, 2005.

Consistency of InfomaxConsistency of Infomax



N. Correa, T. Adali, Y. Li, and V. D. Calhoun, "Comparison of Blind Source Separation Algorithms for FMRI Using a New Matlab Toolbox:

GIFT," in Proc. ICASSP, Philadelphia, PA, 2005.



Clustering of five algorithms using ICASSOClustering of five algorithms using ICASSO

Default mode

Transient task


Left task

Right task

Infomax, FICA1, FICA2, FICA3, JADEInfomax, FICA1, FICA2, FICA3, JADE


Three Review ArticlesThree Review Articles


A Few Software PackagesA Few Software Packages•• The ICA:DTU toolboxThe ICA:DTU toolbox

((http://mole.imm.dtu.dk/toolbox/ica/index.htmlhttp://mole.imm.dtu.dk/toolbox/ica/index.html))•• matlabmatlab

•• three different ICA algorithmsthree different ICA algorithms

•• fMRI specific with demo datafMRI specific with demo data

•• FMRIB Software Library, which includes theFMRIB Software Library, which includes theICA tool MELODICICA tool MELODIC((http://www.fmrib.ox.ac.uk/analysis/research/melhttp://www.fmrib.ox.ac.uk/analysis/research/melodic/odic/):):

•• CC

•• FastICA+FastICA+

•• Complete PackageComplete Package

•• FMRLAB (FMRLAB (http://www.sccn.ucsd.edu/fmrlab/http://www.sccn.ucsd.edu/fmrlab/))•• matlabmatlab

•• infomax algorithminfomax algorithm

•• fMRI specific with additional toolsfMRI specific with additional tools

•• ICALABICALAB•• matlabmatlab

•• multiple ICA algorithmsmultiple ICA algorithms

•• not fMRI specific although one fMRI examplenot fMRI specific although one fMRI exampleincludedincluded

•• GIFT (GIFT (http://icatb.sourceforge.nethttp://icatb.sourceforge.net))••


•• AnalyzeFMRIAnalyzeFMRI((http://www.stats.ox.ac.uk/~marchini/software.hthttp://www.stats.ox.ac.uk/~marchini/software.htmlml))

•• R R

•• FastICAFastICA

•• BrainVoyager(BrainVoyager(http://www.brainvoyager.com/http://www.brainvoyager.com/))•• CommercialCommercial

•• FastICAFastICA

•• Complete PackageComplete Package

•• mat amat a

•• >9 ICA algorithms (more coming) including>9 ICA algorithms (more coming) includinginfomax and fastICAinfomax and fastICA

•• Constrained ICA algorithmsConstrained ICA algorithms

•• Visualization tools and sorting options.Visualization tools and sorting options.

•• Sample data and a stepSample data and a step--by by--step walk throughstep walk through

Group ICA of fMRI Toolbox (GIFT)Group ICA of fMRI Toolbox (GIFT)

LeftHemisphereVisual StimuliOnset

LeftHemisphereVisual StimuliOnset


VoxelVoxel

BOLD Signalhttp://icatb.sourceforge.nethttp://icatb.sourceforge.net

Funded by NIH 1 R01 EB 000840 (to V. Calhoun)Funded by NIH 1 R01 EB 000840 (to V. Calhoun)

Major Contributors:Major Contributors:Tülay Adalı Tülay Adalı – – University of MarylandUniversity of MarylandAndrzej Cichocki, RIKEN, JapanAndrzej Cichocki, RIKEN, JapanJim Pekar Jim Pekar – – Johns HopkinsJohns HopkinsHichem SnoussiHichem Snoussi – – IRCCyN, France

2000+ unique downloads

DemoDemo•• SingleSingle--subject ICAsubject ICA

•• Component EstimationComponent Estimation

•• Defaults fileDefaults file

•• Detrend levelDetrend level

•• ScalingScaling

•• SortingSorting


•• Component Explorer Component Explorer

•• Orthogonal Viewer Orthogonal Viewer

•• Composite Viewer Composite Viewer

•• Examine Regression ParametersExamine Regression Parameters

•• Artifact Removal ToolArtifact Removal Tool


Documents

Lecture09_ICAIntro