Yi Chen, Radoslaw M. Cichy and John-Dylan Haynes

Bernstein Center for Computational Neuroscience Berlin &Charité – Universitätsmedizin

Max-Planck-Institute for Human Cognitive and Brain Sciences, Leipzig

25/11/2011 Berlin

Multi-Scale Mapping Of fMRI Information On The Cortical Surface:

A Graph Wavelet Based Approach

Pattern

Recognition

?

Haxby et al. Science, 2001

Multivariate Pattern Analysis of fMRI Signal

Spatial Range of MVPA Methods

• Haxby et al., Science, 2001 Haynes & Rees, Nature Rev. Neurosci, 2006

Kriegeskorte et al., PNAS, 2007

Whole brain Searchlight techniqueROI-based

Searchlight technique affords unbiased, spatially localized information detection.

Global Local

The fMRI Signal in Space

BOLD changes

Information carried by the fMRI signals resides in the convolved cortical sheets.

• Jin & Kim, Neuroimage, 2008

3D searchlight methods do not take this structural complexity of the brain into account.

The brain has a complex structure:

3D searchlight

Cortical Surface-based Searchlight

• Chen et al., NeuroImage, 2011

Surface-based searchlight respects local geometry

3D searchlight neglects local geometry

Searchlight on cortical surface mesh

Application: Decoding Object Category

Object categories:Trumpets vs Chairs vs Boats

Chen et al., NeuroImage, 2011

Surface-based vs 3D Method

• Chen et al., NeuroImage, 2011

3D method deteriorates spatial specificity

Surface-based method observes local structure

Fusiform gyrus

Collateral sulcus

Fusiform gyrus

Collateral sulcus

Surface-based method localizes fMRI information more precisely

Multiscale Organization of Brain Function

Ocular dominanceand orientation

preference columns

Retinotopic maps Object selective regions

Hierarchical organization with increasing spatial scale

Knowing the spatial scale of patterns is crucial for understanding the brain’s functional organization

Yacoub et al., PNAS, 2008Wandell, Encyclopedia Neurosci., 2007

L R

V1

V2V3

V3

V2

Multiscale Analysis – Wavelet TransformWavelets, or “little waves”, are families of spatially local, band-passing filters:

Information specific to different scales can be extracted with wavelets

Fine scale information

Fine scale wavelet

Large scale wavelet

Output:

Large scale information

Fine scale detail

Large scale detail

Transform

Hackmack and Haynes, in prep.

Scale up

Wavelets on regular grid Wavelets on irregular mesh

Varies on translation Translation invariant

On an irregular mesh, wavelet transform cannot be directly implemented

Wavelets on Irregular Mesh

Another Way to Look at Discrete Fourier Transform

Projecting a signal onto the space spanned by these eigenvectors is thus computationally equivalent to its Discrete Fourier Transform (DFT):

Manipulating the transform coefficients and exploiting the unitary property of U, we can implement filters on the frequency domain. The filtered signal is given by:

)(2

1)(

2

111 iiiii xxxxx Discrete Laplacian:

xKx

)(2

1)(

2

111 iiiii xxxxx

where K is a symmetric matrix, its eigenvectors, when sorted non-decreasingly w.r.t. eigenvalues:

odd is if

even is if

1if

)/2/2cos(/1

)/2/2sin(/1

/1

)(

j

j

j

njhn

njhn

n

hu j

xUx ˆ

matrix diagonal a is where,ˆ~ff

T DxDUx

The diagonal matrix contains the Impulse Response function of the designed filter.fD

Taubin SIGGRAPH '95

For a signal x defined on a one-dimensional, regular and circular field, we have:

Biyikoglu et al., Laplacian Eigenvectors of Graphs, 2007Hammond et al., Applied & Comp. Harmonic Analysis, 2009

Implementing Wavelets via Graph Laplacian

Eigenspectrum

Fre

q. R

espo

nse

Wavelets on irregular mesh can then be defined on the eigenspectral domain:

xy

xy yfxfwxf~

)()()(H

Generalized graph Laplacian H:

characterizes the geometric properties of the graph. The eigenvectors of graph Laplacian have a quasi-frequency property:

xyW

Multiscale Analysis on Irregular Mesh

Fine scale information

Fine scale wavelet

Large scale wavelet

Outputs

Large scale information

Fine scale detail

Large scale detail

Transform

Spectral graph wavelets can be used to achieve multiscale analysis on irregular meshes

Anisotropic Filters on Cortical Surface

Fine scale

Large scale

Vertical Horizontal

Anisotropic filters are possible by using different geometric schemes for the graph Laplacian

Multiscale Analysis of Object Categories/Exemplars

• Cichy et al., Cerebral Cortex, 2011

Exemplars:Child vs Female vs Male

Categories:Objects vs Scenes vs Body parts vs Faces

2-step procedure:

BOLD estimates were sampled onto the cortical surface & transformed with spectral graph wavelets

At each scale, the outputs from the filter banks were taken as feature vectors for classification

Scale Differentiated Analysis ofExemplar and Category Encoding

Large Scale

Fine Scale

Categories Exemplars

Categories are preferentially encoded in large scale and exemplars in fine scale

z-score

Summary

Cortical surface-based method- respects natural geometry of the brain- improves spatial specificity of MVPA

Multi-scale analysis on the cortical surface- can extract information from fMRI signals at different scales using spectral graph wavelets - shows that object categories and exemplars are encoded in different spatial scales in the ventral visual stream

The combination of surface-based technique and multi-scale information mapping promises a better understanding of human brain function

Acknowledgements

John-Dylan Haynes

Fernando Ramirez

NEUROCURE

Jakob Heinzle

Radoslaw M. Cichy

Kerstin Hackmack

Appendix

Spectral Graph Wavelets & Fast Algorithm

Hammond et al., Applied & Comp. Harmonic Analysis, 2009

For filter with compact spatial support, its impulse response function defined on eigenspectral domain needs to be continuously differentiable. Wavelet functions are defined by a family of dilated versions of a single function (mother wavelet).

Mother wavelet needs to meet the admissibility condition.

Fast algorithm is possible by approximating the wavelet function on eigenvalue domain with truncated orthogonal polynomials (e.g. Chebyshev polynomial), and calculating the eigenspace projection with recursive sparse matrix vector multiplications (Sect.6, Hammond et al., 2009).

Note, however, by adopting above fast algorithm, the dilation of mother wavelet is now carried on the eigenvalue domain, rather than the eigenvalue’s rank/index domain.

0)0( ,)(

0

2

gCdxx

xgg

)()( )()()( ,, fpfTfWpgTtg tLtLttt

Multiscale Analysis on Regular Grid

Fine scale detail

Fine scale wavelet

Large scale wavelet

Outputs

Large scale detail

Fine scale detail

Large scale detail

Transform