Detecting activation in fMRI dataKJ Worsley Department of Mathematics & Statistics, and McConnell Brain Imaging Cen-tre, Montreal Neurological Institute, McGill University, Montreal, Canada1

We present a simple approach to the analysis of fMRI data collected from several runs,sessions and subjects. We take advantage of the spatial nature of the data to reduce thenoise in certain key parameters, achieving an increase in degrees of freedom for a mixedeffects analysis. Our main interest is the analysis of the resulting images of test statisticsusing the geometry of random fields. We show how the Euler characteristic of the excursionset plays a key role in setting the threshold of the image to detect regions of the brainactivated by a stimulus.

1 Introduction

1.1 The data and a simple model

Functional Magnetic Resonance Imaging (fMRI) data from a single ‘run’ consists of a timeseries of 3D images collected every ≈3 seconds while the subject is performing a task orreceiving a stimulus inside the scanner. One of the first objectives, which we shall concentrateon here, is to find which areas of the brain are activated by the task or stimulus. An exampleof this is shown in Figure 1.

It should be remembered that fMRI does not directly measure the thing we are mostinterested in – electrical activity of the neurons – but the changes in blood oxygenationindirectly caused by this activity. The fMRI response to a stimulus is delayed and dispersedby about 6 seconds, modelled by convolution of the stimulus with a hemodynamic responsefunction (HRF) (see Figure 2). A simple model at a point s in D-dimensional Euclideanspace (D = 3 here) is a linear model

Y (s) = Xβ(s) + σ(s)ε(s) (1)

where Y (s) is a column vector of n observations at point s, X is a design matrix incorporatingthe response to the task (common to all points), β(s) is a vector of unknown coefficients,σ(s) is an unknown scalar standard deviation, and ε(s) is a column vector of temporallycorrelated Gaussian errors.1

The HRF can be modelled as a gamma function, or difference of two gamma functions,whose parameters may be estimated as well, creating a non-linear model 2−4 give a simplemethod based on a linear model for estimating the delay of the HRF at every voxel. Morecomplex models for the HRF have been estimated by Bayesian methods.5 A ‘balloon’ modelhas been proposed for the HRF and Bayesian methods used to estimate it.6 For simplicitywe shall assume that the HRF is known for the analyses presented here.

1Address for correspondence: KJ Worsley, Department of Mathematics and Statistics, McGill University,805 Sherbrooke St West, Montreal, Quebec, Canada H3A 2K6. E-mail: worsley@math.mcgill.ca. Web:http://www.math.mcgill.ca/keith

1

So far it looks like a straightforward statistical problem of fitting a model for the responseto the task, estimating parameters, then making inference about effects. But what makesfMRI data intriguing is the spatial aspect - each observation Y (s) is an entire 3D image,rather than a single value, and neighbouring voxels (3D pixels) tend to be correlated. It isthis feature that we shall concentrate on here – we shall see that inference for an image oftest statistics involves some deep results in the geometry of random fields.

1.2 Computation issues

Before proceeding further, it is worth pointing out the limitations imposed on our choice ofanalysis by the sheer size of fMRI data. Each 3D image is typically 128 × 128 × 24 voxelsor about 3MB of double precision reals. There are typically 100-200 such images in a run,several runs in a session, several sessions on each subject, and several subjects in a study.Altogether, a study might comprise 100GB of (uncompressed) data. Not a lot by currentstandards, but far exceeding the memory of current computers. We cannot simply read allthe data into R or SAS and perform standard analyses. Special code has to be written todeal with the data piece by piece.

Even then computer time restricts the complexity of analyses: simply averaging the timeseries of one run at each voxel takes several seconds. In some sense we are back to theold pre-computer days and we must look for simple methods (linear rather than non-linearmodels, methods of moments rather than maximum likelihood), and computational shortcuts (e.g. re-using matrix inverses, over-writing data structures) to get the analysis done inreasonable time.

1.3 The spatial component

One simple approach, optimal under certain conditions, is to ignore the spatial componentfor the time being, and carry out separate analyses at each point, that is, fit the model(1) separately at each point s ignoring the neighbours. The parameter estimates may notbe fully optimal, but they will be unbiased, and hypothesis tests will be valid. However atcertain steps in the analysis, it is worth borrowing strength from neighbours to reduce noisein parameter estimates. A simple way of doing this is spatial smoothing by running a 3Dkernel smoother over the parameter image.

Which parameters should we choose for smoothing? The parameters of primary impor-tance are the effects we are interested in (c′β(s) for a contrast vector c), and their standarddeviations. The key quantity for activation detection is their ratio, or T statistic, T (s). Theseparameters are not smoothed, since smoothing always increases bias as the cost for reducingnoise. Smoothing is perhaps best reserved for parameters of secondary importance, such astemporal correlations or other ratios of variances and covariances. This is the approach wetake here.

1.4 Detecting activation

The experimenter often wants to know which regions of the brain are ‘activated’ by the task.This is often the most important, but not the sole goal of the study. Increasingly investigators

2

are interested in other aspects such as the temporal dynamics of the response, and whichareas are co-activated or ’functionally connected’, and how these connections change underdifferent tasks or conditions. Here we shall concentrate on the simple question of detecting‘activation’, a local increase in the effect of the task, with most of the brain unaffected bythe task.

This problem has much in common with outlier detection: a few outlying voxels amongsta large number of unaffected voxels; thresholding a test statistic T (s) at each voxel; problemsof multiple comparisons. It is the latter which we shall concentrate on in Section 3. Theproblem is fascinating because we now have a huge number (≈100,000) of test statisticswhich form a 3D image. Moreover this image T (s) is best regarded as a discrete samplingof a continuous smooth random field (s ∈ <D rather than the lattice of voxels). This hasspurred a recent revival of random field theory that has produced some fascinating newtheoretical work that we shall touch on in Section 3.2.

2 Modelling fMRI data

2.1 Why a univariate analysis

Before fitting the model (1) separately at each voxel, it is worth looking at the conditionsunder which this is optimal. If the design matrix is the same at each voxel (reasonable– the whole brain receives the same stimulus) then separate analyses are optimal if thespatio-temporal correlation structure is separable. This is the case if the variance of all theobservations (written as a single space × times vector) is the Kronecker product of spatialand temporal variance matrices. In particular, the temporal correlations must be the sameat every point in space, and the spatial correlations must be the same at every point in time.There is strong evidence that temporal correlation varies spatially (see Figure 3), though itappears to be roughly constant in grey matter, the regions of most interest. So by fittingthe model separately at each point, we sacrifice a little efficiency, but the analysis is simplerand faster.

2.2 Modelling the temporal correlation

Focusing on the temporal correlation structure, a simple and computationally convenientmodel is the AR(p) model. Fortunately an AR(1) model seems to be adequate for low field(1.5T) fMRI data. We fit this using the old Cochrane-Orcutt method of first estimatingthe mean parameters β(s) by least squares, then the AR(p) parameters via the Yule-Walkerequations. These are modified slightly to reduce bias by equating the observed temporalautocorrelations to their expectations.7 This bias reduction is the first step in a Fisherscoring algorithm to find ReML estimates. Its main advantage is that it is computationallyvery cheap to implement. Spatial smoothing of the AR(p) parameters reduces noise. Wethen pre-whiten the data Y (s) and the covariates X by the AR(p) model (computationallycheap) and re-fit the linear model to obtain estimates β(s) and its variance Var(β(s)) in theusual way. Obviously this model could be better fitted by full ReML, but even the abovesimple method takes 5-30 minutes (depending on the order of the AR model).

3

2.3 A mixed effects model for combining effects across runs, ses-sions and subjects

A great deal of effort goes into analysing fMRI data from one run, but the equally importantproblem of combining these results across runs, sessions and subjects has received less at-tention. Ideally we would like to combine all the fMRI data into a hierarchical mixed effectslinear model, but again the computational cost is prohibitive. Instead we have elected totreat each stage in the hierarchy separately, that is, we take the estimated effects from onestage, e.g. Ei = Ei(s) = c′βi(s) where subscript i refers to run i, as ‘observations’ in a mixedeffects linear model at the next stage:

Ei = z′iγ + σFi εF

i + σRεRi (2)

(dropping s) where zi is a vector of regressors (usually 1 for a simple average, but othercovariates such as run conditions, age and sex (for subjects) might be of interest) and γ = γ(s)is a vector of unknown coefficients. σF

i = σFi (s) = Sd(c′βi(s)) is the unknown ‘fixed effects’

standard deviation from the scan to scan variabiltiy of fMRI data within the same run, andσR

i = σRi (s) is an unknown ‘random effects’ standard deviation from repetitions over runs.

εFi = εF

i (s) and εRi = εR

i (s) are independent standard Gaussian errors.Since the number of time points is high (100–200) then the fixed effects standard deviation

σFi can be accurately estimated from the first level analyses by Sd(c′β(s)) and taken as

fixed. The random effects standard deviation σR is then estimated by ReML using the EMalgorithm (for stability).

One drawback of the EM algorithm is that it always produces positive variance estimates.These can be considerably biased particularly if the true variances are nearly zero. We couldopt for a different method such as Fisher scoring or Newton-Raphson but these are lessstable, something to be avoided if the method is to be repeated at each one of 100,000voxels. Instead a simple re-parameterization by subtracting the minimum (σF

i )2 from each(σF

i )2 and adding it to (σR)2 allows negative estimates of (σR)2 which are much less biased.1

We believe that unbiased estimates of the variance of γ are more important than negativevariance estimates.

2.4 Why a hierarchical analysis

Again we might wonder if we have lost something by splitting up the analysis into stages,keeping only estimated effects and their standard deviations from the previous stage, andthrowing the rest of the data away. To see what happens, first assume that the standarddeviations are known. Provided the runs are independent, then the βi’s are sufficient fortheir parameters, since the residuals contain no further information. If contrasts in βi thatare orthogonal to the contrast of interest c′βi are free, that is not constrained by a modelsuch as (2), then generalised least squares estimates from (2) are optimal. Thus the onlyloss of efficiency comes from doing separate univariate analyses of each contrast of interest,rather than a combined multivariate analysis. Again if the regressors zi and the contrastvariance matrices were identical for each run, the separate univariate analyses would be fullyefficient.

4

2.5 Spatial pooling to increase the degrees of freedom

A more serious problem is the low degrees of freedom of σR(s) at higher stage analyses dueto the small number of runs, sessions and subjects, as small as 2 in some cases. This isaggravated when we look at images of test statistics later on.

Bizarre phenomena occur for images of T statistics with degrees of freedom less than orequal to the number of spatial dimensions. Exact infinities occur at a multitude of pointsin continuous space, caused by exact zeros in the denominator. Although these singularitiesnever coincide with the voxels, they can be arbitrarily close if the image is interpolated.Singularities can form strings or even sheets if the degrees of freedom is 2 or 1, respectively.

To avoid these problems, and to ensure stable standard deviation estimates for analysis atthe next stage, we try to boost the degrees of freedom by a form of local pooling, analogousto pooling the degrees of freedom from separate levels in a one way ANOVA. However theimage of estimated standard deviations σR is far from homogeneous, containing a greatdeal of anatomical structure (see Figure 5) so smoothing this could introduce a lot of bias.Fortunately the ratio of random to fixed effects, σR/σF, where σF is the root mean square ofthe σF

i ’s weighted by their degrees of freedom, has most of the anatomical structure removed.We smooth this with a kernel smoother to reduce noise, then multiply back by σF to get amuch less noisy estimate of σR.

Its effective degrees of freedom (from a Satterthwaite approximation) depends on theamount of smoothing – the more smoothing is applied, the higher the degrees of freedom.Worsley et al. (2002)1 gives a simple formula for this that assumes the errors εF

i , εRi are

independent smoothed white noise. The smoothing kernels for both the white noise and theratio are Gaussians with standard deviations werror and wratio respectively. Then the effectivedegrees of freedom of the ratio is:

νR = νR(2(wratio/werror)2 + 1)D/2,

where νR is the degrees of freedom of the linear model (2). The final effective degrees offreedom ν of σR(s) is estimated by

1/ν = 1/νR + 1/νF,

where νF is the residual degrees of freedom of the fixed effects analysis, equal to the sum ofthe degrees of freedom of the σF

i ’s.Thus the wratio parameter acts as a convenient way of providing an analysis mid-way be-

tween a mixed effects and a fixed effects analysis; setting wratio = 0 (no smoothing) producesa mixed effects analysis; setting wratio = ∞, which smooths the ratio to one everywhere,produces a fixed effects analysis. In practice, we choose wratio to produce a final ν whichis at least 100, so that errors in its estimation do not greatly affect the distribution of teststatistics. This has the extra advantage of allowing us to treat the standard deviations ofsubsequent contrasts in γ as fixed, ready for the same analysis at the next stage of thehierarchy.

2.6 Summary

In summary, we analyse the data in stages, losing some efficiency, but gaining computertime and memory. At the first stage, we fit a simple linear model with AR(p) errors by the

5

Cochrane-Orcutt procedure using spatial smoothing to decrease noise in the AR(p) param-eters. At subsequent stages we take effects of interest as data and fit the simplest form ofmixed effects model using the EM algorithm (for stability) to obtain ReML estimates. Wetake care to obtain unbiased estimates of the variances, at the expense of negative estimatesof variance components. Our main novelty is to increase the degrees of freedom by ‘pool-ing’ variance ratios (not variances) by spatial smoothing, ready for the next stage in thehierarchical analysis.

3 Finding ‘activation’ in fMRI data

We now come to the last step, detecting activation. We look for activation in a search regionS ⊂ <D, usually the whole brain, so that s ∈ S. The simplest method, following the strategyin outlier detection, is to create an image T (s) of test statistics for the activation, then choosea threshold t, and declare as ‘activated’ all points s inside S where T (s) ≥ t. T (s) is justthe effect (contrast of interest) divided by its estimated standard deviation, with ν degreesof freedom, from any stage in the above analysis.

Before proceeding further, it is worth asking if this thresholding strategy is appropriate.As in outlier detection, we expect most of the brain to be unactivated (E(T (s)) = 0), andthe activation confined to a small number of isolated regions where E(T (s)) > 0. Supposeν = ∞ (effectively so by the large number of scans per run, or by spatial pooling). Then itcan be shown that local maxima of T (s) are maximum likelihood estimators of the location ofactivation if the shape of the activation matches the spatial correlation function of the data.8

Furthermore, local maxima of T (s) are likelihood ratio statistics for testing for activation.This suggests that thresholding T (s) is powerful at detecting activation whose extent matchesthe extent of the spatial correlation of the data.

This brings up the interesting question of how much the data should be smoothed beforeanalysis. It is already smoothed by about 6mm by the motion correction procedure. Ideally,we should smooth the data with a kernel whose shape matches the activation to be detected– the Matched Filter Theorem of the image processing literature. Since we do not know theactivation in advance, why not try a range of filter widths, and search over these as well asover the search region itself? This adds an extra dimension to the search regions, known asscale space.9 The price to pay for extra searching, specifically an increase in the threshold t,has been determined by Siegmund & Worsley (1995)8 and Worsley (2001)10.

3.1 Bonferroni and False Discovery Rate

A simple way of controlling the probability of making false discoveries of activation is tomake a Bonferroni correction for the number of voxels in the search region. This is obviouslytoo conservative because of the spatial correlation of the data. The voxels are in any caseartificial – in order to combine data on separate sessions and subjects, it must be interpolatedand re-sampled into a common image space, changing the number of voxels, but not reallychanging the data. In Section 3.2 we model the images as continuous random fields.

An recently proposed alternative11 is to control the False Discovery Rate (FDR), denotedby Q, rather than the probability of ever making a false discovery, denoted by P . This

6

has the advantage that it is unaffected by interpolation and re-sampling, or indeed spatialcorrelation, since it is controlling a rate, rather than an absolute count. As a result thresholdsthat control Q are always a lot lower than thresholds that control P (see Figure 7).

3.2 The Geometry of Random fields

Our main interest in this section is finding an accurate approximation to the P -value of localmaxima inside the search region S. Since these are bounded by the global maximum,

Tmax = maxs∈S

T (s),

the P -value of Tmax gives a conservative P -value for any local maximum. Extensive work hasbeen done on this for the case where the components of ε(s), and hence T (s), are sufficientlysmooth isotropic random fields.12

The main tool that we have used is a concept borrowed from topology, the Euler char-acteristic (EC) of the excursion set. The excursion set At is the set of all points s ∈ <D

where T (s) exceeds a fixed threshold value t (see Figure 7). In 3D, roughly speaking, theEC counts the number of connected components of the excursion set, minus the numberof ‘holes’. For high thresholds the holes disappear and the EC counts the number of localmaxima in the image above the threshold. For even higher threshold values near Tmax, theEC takes the value 1 if the maximum is above the threshold, and 0 otherwise, so that theEC approximates the indicator function for the event Tmax ≥ t. Thus for high thresholdsE{EC(S∩At)} approximates P{Tmax ≥ t}.12 The advantage of the EC is that a simple exactexpression has been found for its expectation when no activation is present:

P{Tmax ≥ t} ≈ E{EC(S ∩ At)} =D∑

d=0

µd(S)ρd(t) (3)

where µd(S) is the d-dimensional intrinsic volume of S and ρd(t) is the d-dimensional ECdensity of T (s). In practice the last term (d = D), first found by Robert Adler,12 is usuallythe most important, and the remaining terms are corrections for when the excursion settouches the boundary of the search region.13

First of all, the intrinsic volumes of a set S in 3D, and their values for a S a ball of radiusr, are

µ0(S) = EC = 1,

µ1(S) = 2× caliper diameter = 4r,

µ2(S) = (1/2)× surface area = 2πr2,

µ3(S) = volume = (4/3)πr3. (4)

The caliper diameter of a convex set is the distance between two parallel tangent planesaveraged over all rotations – for non-convex sets we replace distance by integrated EC of theintersection of S with all parallel planes.

The second component is the EC density which depends on the type of test statistic andthe threshold:

ρd(t) = E{(T ≥ t) det(−Td) | Td = 0}P{Td = 0},

7

where dot notation with subscript d means differentiation with respect to the first d com-ponents of s. It is quite tricky to evaluate this, but results are available for the EC densityof a variety of random fields commonly used as test statistics: the Gaussian random field,12

χ2, t and F random fields.14 For example, for a Gaussian random field,

ρ0(t) = P(T ≥ t),

ρ1(t) = λ1/2 exp(−t2/2)/(2π),

ρ2(t) = λ t exp(−t2/2)/(2π)3/2,

ρ3(t) = λ3/2(t2 − 1) exp(−t2/2)/(2π)2. (5)

3.3 The roughness of the random field

The only unknown parameter is a measure of the roughness of the random field, given by

Var(ε)/Var(ε) = −f(0) = λI, (6)

where f is the spatial correlation function and I is the D×D identity matrix. We shall seehow to estimate λ in the next section. It is interesting to note that the expected EC dependson the spatial correlation only through this parameter λ and nothing else – in other words,the nature of the spatial correlation away from the origin does not affect the expected EC.If for example ε is white noise smoothed with a Gaussian kernel of standard deviation werror

(as in Section 2.5), then f is Gaussian with standard deviation√

2werror and λ = 1/(2w2error).

The approximation (3) appears to be very accurate for low P -values (those usually en-countered in practice) and search regions S of almost any shape or size, even 2D manifoldsembedded in 3D, so that it can be applied to fMRI data mapped onto the cortical surface.In this case µ3(S) = 0 and the d = 2 term is the most important.

There has been considerable recent theoretical work on the EC approximation to theP -value. In the case of T (s) a Gaussian random field and S convex, the approximation (3)is a sum of D terms in decreasing powers of t, plus P{T ≥ t}. It had been conjectured8 thatthese were the first D terms in a power series expansion for P{Tmax ≥ t}. This was basedon a completely different approach developed by David Siegmund and his co-workers thatused volumes of tubes. Here the random field is approximated by a finite Karhunen-Loeveexpansion, and the P -value for its maximum is then the volume of a particular tube aboutthe search region, which can be evaluated using Weyl’s (1939) tube formula. Takemura andKuriki (2002)15 have proved this conjecture when the expansion is finite. Using results ofPiterbarg, Robert Adler has shown that the expected EC is an even more precise P -valueapproximation than previously thought: the error is exponentially (not just polynomially)smaller than the smallest term in the expected EC expansion.16

3.4 Non-isotropic random fields

The above theory assumes that the random field is isotropic, in particular that the widthof the smoothing kernel werror is constant everywhere. This is rarely the case; fMRI data issmoother in grey matter than white matter, for example (see Figure 6). We need new theoryto cover this case. A simple heuristic, borrowed from the geostatistics literature, is to try

8

to warp the brain to a new set of coordinates so that it becomes more nearly isotropic.17 Todo this, we must stretch the rough spots where werror is small, and shrink the smooth spotswhere werror is large. What is required is a form of local multidimensional scaling, where thedistance between lattice points is inversely proportional to werror. Once this is done, we canrecalculate the intrinsic volumes in the new space and apply the formula (3).

To do this. we need a local estimate of werror, or equivalently λ1/2. This is straightforward.Let r be the vector of pre-whitened residuals from the linear model (1), and let u = r/(r′r)1/2

be the vector of normalised residuals. It can be shown that

λ1/2 = (u′u)1/2, (7)

where u is the numerical derivative of u along the edge of the lattice of voxels, is unbiasedfor λ1/2 (Worsley et al., 1999).18 Figure 6 shows an example of the corresponding werror.

Why stop at D = 3 dimensions? Why not increase the number of dimensions of theembedding space to get a better approximation to isotropy? A glance at (7), combined withthe realisation from (6) that λ1/2 is the standard deviation of the derivative of the errors,shows that exact sample isotropy can be achieved if the new coordinates are the normalisedresiduals in D = n dimensions. In other words, we simply replace the 3D coordinates s bythe nD coordinates u.

Finding the intrinsic volumes in the nD coordinate space is straightforward in principle,but tricky in practice. We must divide the transformed search region S into tetrahedra,triangles, edges and points, find the intrinsic volumes of each, and combine them in aninclusion-exclusion formula.18 Fortunately, the most important d = D term is easiest tocalculate: the ‘volume’ of S in the transformed space is:

µD(S) =∫

S1 =

∫

Sdet(u′u)1/2,

where now u is an n × D matrix of numerical derivatives of u with respect to s. Again itcan be shown to be unbiased.18

A ball is a lower bound for any search region of the same volume, quite accurate if S isnot too concave, so an expedient approximation is to equate µD(S) to the volume of the ball,solve for the radius, then find the lower order intrinsic volumes (using e.g. (4) for D = 3).We then insert these estimated intrinsic volumes into (3), setting λ = 1 in the EC densities(5).

We might be concerned that this approach will not work in the limit. There is no concernabout what happens as the lattice becomes finer, provided the random fields are smooth. Themain concern is what happens as the sample size n becomes infinte, because then the numberof dimensions of the embedding space also becomes infinite. However it can be shown, usingthe famous Nash Embedding Theorem, that the number of dimensions required to achieveexact isotropy is finite, in fact less than or equal to D(D + 1)/2 + D. The rigor of this hasbeen firmly established by Taylor & Adler (2003),19 who have taken a more fundamentalapproach by generalising Robert Adler’s work to random fields on manifolds.

3.5 Summary

This section looks at methods of finding activation in a smooth image of test statistics.The P -value at high thresholds is approximated by the expected EC, which has an exact

9

expression for any threshold. The approximation is best when the search region is not tooconcave – for a highly convoluted search region such as the cortex, it might be better toenclose it in a convex hull first. The parameters in this expression, the intrinsic volumesof the (transformed) search region, can be estimated from the normalised residuals of thelinear model that generated the test statistic. Results are available for all the commontest statistics, including Hotelling’s T 2, which has been used for finding differences in brainshape.20

The method is quite general and can be applied in almost any problem (not just fMRI)in any number of dimensions, even when the search region is a 2D surface embedded in3D. In 1D, the same techniques can be used in Functional Data Analysis to search for saydifferences in curves.21

Finally, Bonferroni should not be abandoned altogether. The random field theory criticalthresholds, found by equating (3) to say 0.05 and solving for t, are sometimes larger thanthe Bonferroni threshold, particularly if the distance between voxels is large or the degreesof freedom is small. There is no contradiction here – Bonferroni is conservative for searchingat the voxels, whereas (3) searches over the continuous space between the voxels as well.In practice our software always takes the minimum of the Bonferroni and random fieldthresholds.

4 Application

The methods described in this paper were applied to an fMRI experiment on pain perception.22

The idea was to find which areas of the brain were activated by the perception of pain, asopposed to touch, so the effect of interest was the difference between a painful heat stimulusand a neutral heat stimulus.

After 9 seconds of rest, a subject was given a painful heat stimulus (49oC) to the leftforearm for 9 seconds, followed by 9 seconds of rest, then a warm stimulus (35oC) for 9seconds, repeated 10 times for 6 minutes in total (see Figure 2). During this time thesubject was scanned every 3 seconds (120 frames) using a Siemens 1.5 T machine and 12slices of 128 × 128 pixel BOLD images were obtained (2.3 × 2.3 × 7mm voxel steps). Thefirst 3 frames were discarded, leaving n = 117 frames as data.

Two ‘box’ functions were created for the hot and warm stimuli then convolved with theHRF (modelled as the difference of two gamma functions) and sampled at the slice acquisitiontimes (Figure 2). Four covariates for a cubic in the scan time, to allow for drift (see Figure1), were added to create the design matrix X. The contrast of interest was c = (1 −1 0 0 0 0)for comparing the hot stimulus (1) with the warm stimulus (-1), ignoring the 4 coefficientsof the cubic drift (0 values).

Only part of the data is analysed here – 4 runs in one session on one subject. Ouranalysis differs from that in Worsley et al. (2002)1 in that the data here are converted topercentages of the whole brain before analysis – this was found to substantially decreasethe noise. Figure 3 shows the (bias reduced) estimated coefficients a1, a2, a3, a4 of an AR(4)model fitted to the residuals from one run (run 2) – clearly an AR(1) model is adequate, sothis was used for the rest of the analyses. The resulting effects, standard deviations and Tstatistics (117-6=111 degrees of freedom) for the hot - warm contrast are shown in Figure 4.

10

There seems to be strong and consistent evidence for activation near the centre of the slice.The 4 run effects were combined using zi = 1 in the mixed effects model (2) – the

parameter of interest is then the common effect γ. Figure 5 shows how the mixed effectsstandard deviation was calculated by smoothing the ratio of the random to fixed effectsstandard deviation which removes nearly all the anatomical structure. After smoothing witha 20mm FWHM (Filter width in image processing is usually measured by its Full Widthat Half Maximum (FWHM). For a Gaussian filter with standard deviation w, FWHM =

w√

8 loge 2.) Gaussian filter (wratio = 8.5mm), the ratio is close to one both inside and

outside the brain (indicating no evidence for a random effect), except for two anterior andposterior regions where there is about 30% extra standard deviation due to random effects.Multiplication back by the fixed effects standard deviation gives the final mixed effectsstandard deviation with about ν = 111 degrees of freedom (νR = 4− 1 = 3, νF = 4× 111 =444, werror = 3.4mm), large enough to be treated as infinite. This standard deviation,together with the estimated effect γ and the T statistic are shown in the last column ofFigure 4. There is now much stronger evidence for an effect in the centre of the slice.

To threshold the image of T statistics we first need to assess the smoothness of the image.Figure 6 shows the estimated werror (in FWHM units) of the Gaussian smoothing kernel bothover scans (from the 117 residuals of model (1)) and over effects (from the 4 residuals ofmodel (2)). The effects FWHM is much noisier then the scans FWHM because it is basedon far fewer residuals. After smoothing the effects FWHM, both show similar patterns of≈6mm smoothing in white matter areas and outside the brain (due to the 6mm smoothingapplied during motion correction) and increased smoothness of ≈10mm in grey matter areas.This is why we took the average cerebral smoothing as 8mm for werror in the calculation ofν.

The next step is to define the search region S. We took the first scan of the fMRI data andthresholded it at 450. The estimated roughness integrated over the search region gives anestimated intrinsic volume (in isotropic space) of µ3(S) = 7589 (unitless). The other intrinsicvolumes can be found either by the spherical approximation (µ0...2(S) = 1, 48.8, 933.8) or bythe more accurate but more complicated tetrahedral lattice (µ0...2(S) = 1,−821.1, 877.0).Both methods yield roughly the same P = 0.05 threshold, found by equating (3) to 0.05and solving for t: 5.08 and 5.09, respectively. However the Bonferroni threshold was smaller:t = 4.90, because there were only 12 slices in this data set so only 30786 voxels in the searchregion. The final thresholded data is shown in Figure 7, together with the much lower FalseDiscovery Rate Q = 0.05 threshold of t = 2.66.

5 Discussion

5.1 Other aspects of fMRI data analysis

There are many aspects of fMRI data analysis that we have ignored in this paper. ThefMRI data is in fact the modulus of the (complex) Fourier transform of data generated bythe scanner, so their might be some additional information in the phase that is presentlyignored. The fMRI data is corrected for subject motion in the scanner resulting in 6mmsmoothing. The fMRI data ‘drifts’ over time, requiring trend terms in the linear model, or

11

better still, a state-space model. The fMRI data is collected in 2D slices at slightly differenttimes, so the response and hence the design matrix X must be slightly different for differentslices. The hemodynamic response is quite complicated. It is not quite additive for closelyspaced stimuli, nor is the HRF quite constant spatially or temporally. A more sophisticatedapproach is to fit a proper hemodynamic model such as the ‘balloon’ model.6

Most fMRI experiments are repeated in different sessions and subjects, so great caremust be taken to align or register data from the same subject in different sessions anddifferent subjects. First an anatomical scan must be acquired on each subject immediatelybefore or after the functional scans, then intensity corrected for non-uniformity, then alignedusing either linear or non-linear transformations to a common atlas standard. The sametransformations are then applied to the functional data after registering the functional scanswith the anatomical scan. Although there are many common anatomical features in brainanatomy, there can be huge differences at a smaller scale, so exact alignment is impossibleand some compromises must be made.

Since the neuronal response only occurs in grey matter, it might seem preferable toproject all the data onto the cortical surface and carry out all the analysis (model fitting,smoothing, activation detection) in this 2D manifold, a procedure known as Cortical SurfaceMapping.23 However there are considerable technical obstacles to overcome, such as fitting atriangular mesh to the cortical surface from the anatomical scan and registering it with thefunctional scans.

5.2 The future: connectivity and EEG/fMRI fusion

So far we have concentrated solely on finding activation of the brain to a stimulus. What isoften of more interest is discovering which parts of the brain are functionally ‘connected’, inother words, how information is transmitted between different parts of the brain. Principalcomponents is very useful at exploring these correlations. Cao & Worsley (1999)24 havelooked at the 6D random field of correlation coefficients between all pairs of voxels andapplied random field theory to determine a threshold for finding pairs of ‘co-activated’ voxels.For more sophisticated modelling, the number of possible connections between all pairs ofvoxels greatly exceeds the amount of data available, so some researchers have isolated a smallnumber of regions of interest and applied methods such as path analysis and simultaneousequations.

The main obstacle is that fMRI data can only measure brain activity every few seconds,which is far too slow to discover causal connectivity at the neuronal level. EEG has a muchfiner temporal resolution, of the order of milliseconds, but trying to infer the spatial distri-bution of this signal from electrodes on the outside of the head seriously limits the spatialresolution. The ideal would be to combine the two modalities. Many groups are working onthis, but there are enormous technical difficulties of simultaneous EEG recording while thesubject is in the MR scanner.25 Once these hurdles have been satisfactorily overcome, therewill be a very challenging statistical problem of how to fuse the spatial resolution of fMRIwith the temporal resolution of EEG to uncover the true functioning of the brain.

12

References

1 Worsley KJ, Liao C, Aston JAD, PetreV, Duncan GH, Morales F, Evans AC.A general statistical analysis for fMRIdata. NeuroImage 2002; 15: 1-15.

2 Lange N, Zeger SL. Non-linear Fouriertime series analysis for human brain map-ping by functional magnetic resonanceimaging (with Discussion). Journal ofthe Royal Statistical Society, Series C(Applied Statistics) 1997; 14: 1-29.

3 Purdon PL, Solo V, Weisskoff RM, BrownE. Locally regularized spatiotemporalmodeling and model comparison for func-tional MRI. NeuroImage 2001; 14: 912-23.

4 Liao C, Worsley KJ, Poline J-B, DuncanGH, Evans AC. Estimating the delay ofthe response in fMRI data. NeuroImage2002; 16: 593-606.

5 Genovese CR. A Bayesian time-coursemodel for functional magnetic resonanceimaging data (with Discussion). Jour-nal of the American Statistical Associ-ation 2000; 95: 691-719.

6 Friston KJ, Mechelli A, Turner R, PriceCJ. Nonlinear responses in fMRI: TheBalloon model, Volterra kernels and otherhemodynamics. NeuroImage 2000; 12:466-77.

7 Worsley KJ. Comment on ‘A Bayesiantime-course model for functional mag-netic resonance imaging data’ by C. Gen-ovese. Journal of the American Statis-tical Association 2000; 95: 691-719.

8 Siegmund DO, Worsley KJ. Testing for asignal with unknown location and scalein a stationary Gaussian random field.Annals of Statistics 1995; 23: 608-39.

9 Poline J-B, Mazoyer BM. Enhanced de-tection in brain activation maps usinga multifiltering approach. Journal ofCerebral Blood Flow and Metabolism 1994;14: 639-42.

10 Worsley KJ. Testing for signals with un-known location and scale in a χ2 ran-dom field, with an application to fMRI.Advances in Applied Probability 2001;33: 773-93.

11 Genovese CR, Lazar NA, Nichols TE.Thresholding of statistical maps in func-tional neuroimaging using the false dis-covery rate. NeuroImage 2002; 15: 772-86.

12 Adler RJ. The Geometry of RandomFields. New York: Wiley, 1981.

13 Worsley KJ. Boundary corrections forthe expected Euler characteristic of ex-cursion sets of random fields, with anapplication to astrophysics. Advancesin Applied Probability 1995; 27: 943-59.

14 Worsley KJ. Local maxima and the ex-pected Euler characteristic of excursionsets of χ2, F and t fields. Advances inApplied Probability 1994; 26: 13-42.

15 Takemura A, Kuriki S. On the equiva-lence of the tube and Euler character-istic method for the distribution of themaximum of Gaussian fields over piece-wise smooth domains. Annals of Ap-plied Probability 2002; 12: 768-96.

16 Adler RJ. On excursion sets, tube for-mulae, and maxima of random fields.Annals of Applied Probability 2000; 10:1-74.

17 Sampson PD, Guttorp P. Nonparametricestimation of nonstationary spatial co-variance structure. Journal of the Amer-ican Statistical Association 1992; 87: 108-19.

18 Worsley KJ, Andermann M, Koulis T,MacDonald D, Evans AC. Detecting changesin nonisotropic images. Human BrainMapping 1999; 8: 98-101.

19 Taylor JE, Adler RJ. Euler character-istics for Gaussian fields on manifolds.

13

Annals of Probability 2003; 31: 533-63.

20 Cao J, Worsley, KJ. The detection of lo-cal shape changes via the geometry ofHotelling’s T 2 fields. Annals of Statis-tics 1999; 27: 925-42.

21 Ramsay JO, Silverman B. FunctionalData Analysis. New York: Springer,2002.

22 Chen J-I, Ha B, Bushnell MC, Pike B,Duncan GH. Differentiating noxious- andinnocuous-related activation of humansomatosensory cortices using temporalanalysis of fMRI. Journal of Neurophys-iology 2002; 88: 464-74.

23 Andrade A, Kherif F, Mangin J-F, Par-adis A-L, Worsley KJ, Simon O, De-haene S, Le Bihan D, Poline J-B. De-tection of fMRI activation using corticalsurface mapping. Human Brain Map-ping 2001; 12: 79-93.

24 Cao J, Worsley KJ. The geometry ofcorrelation fields, with an applicationto functional connectivity of the brain.Annals of Applied Probability 1999; 9:1021-57.

25 Dale AM, Liu AK, Fischl BR, BucknerRI, Belliveau JW, Halgren E. Dynamicstatistical parametric mapping: com-bining fMRI and MEG for high-resolutionimaging of cortical activity. Neuron 2000;26: 55-67.

14

0

500

1000First scan of fMRI data

(a)

(b)(c)

−5

0

5

T statistic for hot − warm effect

(a)

(b)(c)

0 50 100 150 200 250 300 350

870

880

890 hot

rest

warm

(a) Highly significant effect, T=6.5888

0 50 100 150 200 250 300 350

790

800

810

820hot

rest

warm

(b) No significant effect, T=−0.73591

0 50 100 150 200 250 300 350

790

800

810

(c) Drift

Time, seconds

Figure 1: One slice of the first scan of the fMRI data (front of the brain on top) whilethe subject received an alternating hot and warm stimulus, interspersed with rest peri-ods. The corresponding image of T statistics (111 df) for testing for the effect of hot -warm stimuli is shown below. On the right are the time courses at 3 pixels chosen to illus-trate (a) a highly significant effect, (b) no significant effect, and (c) drift. For colour, seehttp://www.math.mcgill.ca/keith/smmr/smmr.pdf

15

0 50 100 150 200 250 300 350−1

0

1

2(a) Alternating hot and warm stimuli separated by rest (9 seconds each).

hot

warm

hot

warm

0 20 40 60−0.2

0

0.2

0.4

(b) Hemodynamic response function: difference of two gamma densities

0 50 100 150 200 250 300 350−1

0

1

2(c) Responses = stimuli * HRF, sampled every 3 seconds

Time, seconds

Figure 2: The covariates for the linear model generated by convolving ‘box’ functions for thehot and warm stimuli (a) with the hemodynamic response function (b) sampled at the sliceacquisition times (c). For colour, see http://www.math.mcgill.ca/keith/smmr/smmr.pdf

a1

a2

a3

−0.1

0

0.1

0.2

0.3

a4

Figure 3: The coefficients of an AR(4) model fitted to the residuals from the linearmodel, bias reduced and smoothed, for one slice. Temporal correlation is higher in greymatter than white matter. An AR(1) model appears to be adequate. For colour, seehttp://www.math.mcgill.ca/keith/smmr/smmr.pdf

16

Run 1 Run 2 Run 3 Run 4

Effe

ctS

d of

effe

ctT

= E

ffect

/ S

d

−1

0

1 Combined

0

0.1

0.2

−5

0

5

Figure 4: Effects (in percentage units), their standard deviations, and T statistics for the4 runs (left), together with their combined effect from the mixed effects model (right). Forcolour, see http://www.math.mcgill.ca/keith/smmr/smmr.pdf

17

Random effects sd, 3 df Fixed effects sd, 444 df

0

0.05

0.1

0.15

0.2

Mixed effects sd, ~111 df

Random sd / fixed sd

0.5

1

1.5

Smoothed sd ratio

Figure 5: Random, fixed and mixed effects standard deviations (top). The mixed ef-fects standard deviation is found by smoothing the ratio of random / fixed effects (be-low), then multiplying back by the fixed effects. The mixed effects degrees of freedom of≈111 is between the random (3) and fixed (444) effects degrees of freedom. For colour, seehttp://www.math.mcgill.ca/keith/smmr/smmr.pdf

FWHM of scans FWHM of effects

0

5

10

15

20FWHM of effects (smoothed)

Figure 6: Estimated smoothness of the scans (left) and effects (middle, and smoothed,right) in FWHM units (mm). Grey matter regions are smoother than white matter regions.Outside the brain the smoothness is ≈6mm due to the 6mm smoothing applied during motioncorrection. For colour, see http://www.math.mcgill.ca/keith/smmr/smmr.pdf

18

Figure 7: T statistic image for the hot - warm effect, combined over 4 runs (≈111 df).The same slice as in the bottom right of Figure 4, cropped to the brain, is shown cut-ting the mid-cortical surface (top left). The left anterior is facing the camera. The re-maining 3 figures, from top to bottom, left to right, show the search region S (trans-parent) and the excursion sets At (blue) for thresholds: t = 1.65 (P = 0.05, uncor-rected, EC(S ∩ At) = 42, expected=75.9), t = 2.66 (False Discovery Rate Q = 0.05,EC(S ∩ At) = 55, expected=39.2), t = 5.09 (P = 0.05, corrected for searching over S,EC(S ∩ At) = 16, expected=0.05). Note that for high thresholds the holes in the excursionset disappear, and the EC counts the number of connected components. For colour, seehttp://www.math.mcgill.ca/keith/smmr/smmr.pdf

19