1

Spatio-temporal correlations in fMRI time series: the whitening approach

1,2Bosch-Bayard, J.; 3Riera-Diaz, J.; 4Biscay-Lirio, R.; 2Wong, K.F.K.; 5Galka, A.; 6Yamashita, O.; 7Sadato, N.; 3Kawashima, R.; 1Valdes-Sosa, P., 8Miwakeichi, F.; 2Ozaki, T. 1Cuban Neuroscience Center, Havana, Cuba 2The Institute of Statistical Mathematics, Tokyo, Japan 3IDAC, Tohoku University, Sendai, Japan 4Inst. of Cybernetics, Mathematics and Physics, Havana, Cuba 5Institute of Experimental and Applied Physics, University of Kiel, Germany 6ATR Computational Neuroscience Labs, Kyoto, Japan 7National Institute for Physiological Science, Okazaki, Japan 8Chiba University

Keywords: fmri, time series, NN-ARx, causality, AIC, connectivity, whitening, innovations

Abstract

For the purpose of statistical characterization of the spatio-temporal correlation

structure of high-dimensional fMRI time series we propose an innovation approach,

based on whitening the data by nearest-neighbors autoregressive modeling with

external inputs (NN-ARx). It is demonstrated that the whitening step is suitable for

elucidating the significant instantaneous spatial dependences between remote voxels.

Potential and limitations of characterizing causality by autoregressive modeling are

discussed in the context of the study of brain connectivity structure. Results for the

analysis of fMRI data recorded during visual and motor stimulus experiment are also

shown.

2

1. Introduction The method of functional Magnetic Resonance Imaging (fMRI) using Blood Oxygen

Level-Dependent (BOLD) contrast has a very high resolution to localize

hemodynamics activation in space, which has been exploited well by using

appropriate statistical methods (Friston et al., 1995, Penny and Friston, 2003). Even

so the relevance of these advances, in the last few years, there is an increasing

interest in understanding the interactions and mutual relationships between brain

regions when the brain displays task-related activation. Recently, fMRI time series

have been suggested as a source of information for the analysis of the patterns of

brain connectivity. Following this suggestion, several statistical methods have been

introduced in order to analyse the correlation structure in fMRI data (McIntosh and

Gonzales-Lima, 1994; Buechel and Friston, 1997). In a certain sense, these methods

have to be regarded as static, because the dynamic nature of the correlations in fMRI

time series are still ignored, and only zero-delay (i.e. instantaneous) correlations are

considered. Additionally, in those studies the directions of causality between pairs of

regions are pre-specified by the analyst. Later, several authors have generalized

these statistical approaches to include some type of dynamics properties (Harrison,

2003; Goebel et al., 2003; Dodel et al., 2002, Lahaye et al., 2003; Yamashita et al.,

2005), such that both temporal and spatial correlation structures are being estimated

directly from the data.

Two important problems arise when attempting to characterize spatial and temporal

correlation structure from fMRI time series, the dimensionality problem and the

aliasing problem. The first one comes from the high spatial resolution of fMRI images,

which renders it infeasible to consider all possible pairs of voxels explicitly. In order to

obviate this problem usually one of the following approaches is employed: Either

special reference voxels or regions of interest are chosen in advance, or the data is

modified by averaging or parcellation procedures (Lahaye et al., 2003). The second

problem, i.e. the aliasing problem, has not been paid much attention to in the past,

although it may easily cause confusion in the study of brain connectivity. When we try

to estimate the dynamical properties behind the observed time series, the sampling

rate needs to be chosen properly. If the speed of change of the dynamical processes

is very fast and the sampling frequency is not sufficiently high, it has to be expected

that aliasing problems result (well-known, for example, from wheels of carriages in

movies apparently rotating in reverse direction). There is a particular problem with the

3

study of connectivity from fMRI data, since the concept of connectivity relates to

dynamic activity at the level of neural events, while fMRI data represents slowly

changing hemodynamic BOLD signals, which may reflect only indirectly the

underlying dynamic neural activations, measured with a sampling time of a few

seconds. In general, it is impossible to estimate spectral properties of dynamics on

the millisecond level from time series sampled at sampling times of a few seconds.

This becomes possible only by introducing special assumptions about the dynamics

(Riera et al. 2005).

Notwithstanding, fMRI time series are not completely random in space and time.

They display definite patterns of spatio-temporal correlations, and it seems

reasonable to presume that these correlations are caused by the following two

reasons. First, neural events activated by some task may, besides fast dynamics,

also produce slow dynamics in the BOLD signals. Second, sequences of fast

dynamic neural events in separate brain regions with distinct time order may be

causing hemodynamic activities with time delay, but since they are measured with

large sampling time, the small delays can be neglected, such that they appear to

occur almost instantaneously. If we want to statistically characterize the dynamics of

neural connectivity from fMRI time series, it seems that we cannot escape the difficult

problem of finding a specific realistic dynamic structure connecting the fast neural

dynamics and the slow hemodynamics, represented by the fMRI time series (see

Friston, 2000; Riera et al., 2004a). Although the statistical characterization of spatio-

temporal correlations in fMRI time series is not equivalent to the statistical

characterization of underlying connectivity, the former may provide us with useful

tools to obtain insight and evidence on the latter, provided we approach the problem

of finding a dynamical model explaining the interaction of fast neural dynamics and

slow hemodynamics.

In the present paper, we confine ourselves to the problem of statistical

characterization of the spatio-temporal correlation structure of fMRI data in the

original full-dimensional time series and introduce a new practical procedure to be

employed in future studies of connectivity. In section 2.1, we briefly describe the NN-

ARx method and show how most spatio-temporal correlations in high-dimensional

fMRI time series can be removed by NN-ARx modeling, thereby transforming the

data into nearly white innovations. While correlations between neighboring voxels are

removed, correlations between remote pairs will remain present, which enables us, in

4

section 2.2, to formulate a method for correlation mapping. Given a reference voxel,

this method provides an exploratory method for finding correlated pairs of remote

voxels. From there, in section 2.3, we generalize this concept to correlations between

regions, as a global measurement of connectivity. Also provide a statistical test for

the significance of correlations. In Section 3 we present the results of regions

connectivity analysis in two different experimental conditions: a visual (section 3.1)

and a motor (section 3.2) task in a group of 10 subjects for each experiment. In

section 4.1 and 4.2 we discuss these results. In section 4.3 we discuss the

implications of causal modeling in time series analysis as well as implications of the

presented method for the study of connectivity. We then review the concept

“innovation approach” in section 4.4, developed by N. Wiener (see Kailath, 1974).

Finally in section 4.5 we present some limitations in the application of this

methodology in its current stage.

2. Method

2.1 NN-ARx Model -again

Functional magnetic resonance imaging (fMRI) data represents spatial-temporal

measurements of the hemodynamic activity in the brain; the data is given as vector

time series of extraordinarily large dimension. A simple way of whitening such spatio-

temporal data consists of introducing a causal spatio-temporal dynamical model, i.e.

removing the spatial and temporal correlations from the data and generating the prediction error for each voxel. For example, for a voxel at spatial position ( , , )v i j k=

the prediction error at time point t is given by

* * *1 1 2| , ,...,v v v

t t t t t t py E y y y yε − − − −⎡ ⎤= − ⎣ ⎦ , (1)

where by * * *1 2( , ,..., )t t t py y y− − − the set of values at all voxels at time point t l− shall be

denoted. If in addition an external time-dependent stimulus, given by

, 1,...,t d t d t d rS S S− − − − − is to be used for the prediction of vty , the prediction error will be

given by

( ) * * *1 1 2 , 1,...| | , ,..., , ,v v v

t t t t t p t d t d t d rt y S y E y y y y S S Sε − − − − − − − − −⎡ ⎤= − ⎣ ⎦ , (2)

5

A simple linear model that summarizes this approach was introduced in Riera et al.,

2004b) as the Nearest Neighbor with an eXogenous variable (NN-ARx), which is

formulated as follows:

1 0

p rvv v v v v v v

t t k t k t k t k d tk k

y y sXμ φ ξ θ ε− −Δ − −= =

= + + + +∑ ∑ , (3)

where:

0

v v kt k

kt

δ

μ γ=

=∑ is intended to gather the potential drift of the signal. In our case, this term

is not modeled as a constant mean but as a polynomial, which grade is also a model

parameter to be estimated.

1

pv vk t k

kyφ −

=∑ is the autoregressive term that describes the hemodynamics of the voxel

itself.

The vector '

{ , ' }vv

vvvX χ= ∈Ω and vΩ is the region that contains the nearest

neighbors of voxel v .

The fourth term on the right hand side of the NN-ARx model represents the

hemodynamic response to the stimulus. Of course we may assume the existence of

a delay of the effect of the stimulus on the output by introducing lag d in 0

rvk t k d

ksθ − −

=∑ .

If we plot this term for each time point as a spatial map for all voxels, we will see,

“when”, “how” (negative or positive, strong or weak) and “where” in the brain the

response to the stimulus takes place (see Riera et al., 2004b).

In other words, the one-step-ahead prediction for the voxel at position v is given as a

linear combination of its own past, the past of its nearest neighbors, the past of the

stimulus and a trend component.

Estimates of the parameters of this model are obtained by solving a linear equation,

by means of a least square procedure.

It must be emphasized that in general, it is possible the presence of instantaneous

correlations between the dynamical noise terms driving neighbouring voxels, i.e. we

have tacitly assumed that within the high-dimensional AR model the noise covariance

6

matrix 'v vvt tEε ε ε⎡ ⎤= ⎣ ⎦Σ was diagonal. This may not be an appropriate assumption, if

the NN-ARx model is intended to be a general model for spatio-temporal dynamics.

One key feature of NN-ARx is the use of the instantaneous Laplacian operator L for

removing instantaneous correlations between the neighbouring voxels. The Laplacian

operator can be defined as follows:

'

'

1v vv vt t vt

v v

LGy x x χ

∈Ω= = − ∑ (4)

where G is the number of voxels in vΩ and vtx is the time series data before

applying the Laplacian operator.

In fact, instead of multiplication by 1G

, the factor to be multiplied may be chosen as a

model parameter using minimization procedure by AIC.

Even though the covariance matrix of the noise vtε in the transformed space may be

assumed to be diagonal ( 2'v vE Int tε ε σ⎡ ⎤ =⎣ ⎦ ) the covariance matrix of the

untransformed noise ( 1v vL ttξ ε−= ) will be non-diagonal and given by

( ) 1'2'v vE LLnt tξ ξ σ−

⎡ ⎤ =⎢ ⎥⎣ ⎦.

This approach represents a simple but useful way of characterizing a spatially

homogeneous instantaneous correlation between the noises of neighboring voxels

and was used for the first time by Galka et al. (2004) and Yamashita et al. (2004) for

the purpose of estimating dynamical inverse solutions from EEG time series. By

fitting the NN-ARx model both with and without employing the Laplacian operator to

the same fMRI data set, and comparing the resulting values of AIC, it can be

confirmed that employing the Laplacian provides a superior modeling.

The parameters of the model: { , , , , }v v v vXk k v kφ θ σ γ have to be estimated for each

voxel from the BOLD signals. The model selection consists of determining both the

model orders and the delays. These magnitudes are denominated the global

7

parameters of the model and are comprised in the vector

( , , , , )p r d δΛ = Δ (5)

The model selection can be a time consuming process, so that previous selection of

feasible range of values inside which each parameter can be varying is a helpful

procedure to save time. This selection has to be made under the basis of proper

knowledge about the nature of the experiment itself, sampling rate of fMRI, etc.

2.2. Voxelwise Innovations and Correlation Maps

Connectivity analysis of brain function can be focused in two different ways: one way

is to study the connections between some few and very local areas that have been

previously selected by some way, which we call the voxelwise approach. The second

way is to try to elucidate global connectivity maps between different areas of the

brain under some experimental condition, that we call the global connectivity

approach or regions connectivity. Regions connectivity can be a first step before

going to a more precise voxelwise approach and can be useful to select those voxels

or small areas that could be interesting for a voxelwise analysis. In this section we

introduce our approach to the voxelwise connectivity.

In the previous sections it has been shown how spatio-temporal correlations between

a given voxel and its neighbors, as well as influences from the external stimulus, are

removed from the corresponding transformed fMRI time series v

ty by the model

given by Eq. 3; the resulting innovations are given by

1 0

p rvv v v v v v v

t t t k t k t k t k dk k

y y sXε μ φ ξ θ− −Δ − −= =

⎛ ⎞= − + + +⎜ ⎟

⎝ ⎠∑ ∑ (6)

It seems unlikely that the hemodynamic activity in one voxel will directly affect the

hemodynamic activity of remote voxels within a few second (i.e. the sampling time).

However, the neuronal electrical signals transfer much faster than hemodynamic

activities and may reach remote voxels within a millisecond, i.e. almost instantaneous

with respect to the typical time scale of the hemodynamics. Furthermore, two distant

voxels with a special neuronal connection may simultaneously receive a large signal

input coming from a common source, whereby the prediction errors of these two

voxels become strongly correlated. The Laplacian operator was designed to remove

8

instantaneous spatial correlations only between neighboring voxels, but not between

distant voxels.

The NN-ARx model provides us with a powerful, yet computationally simple tool for

the purpose of finding this type of instantaneous connectivity between remote voxels. Let the innovations for a pair of voxels v and w of be given by

( )1 , 1,...| ,..., , , ,v v v v v v

t t t t p t d t d t d rtt y E y y y S S Sξε − − − − − − −−Δ⎡ ⎤= − ⎣ ⎦ (7)

and

( )1 , 1,...| ,..., , , ,w w w w w w

t t t t p t d t d t d rtt y E y y y S S Sξε − − − − − − −−Δ⎡ ⎤= − ⎣ ⎦ (8)

respectively. If w and v were nearest neighbors, we would not expect vtε and w

tε to be

strongly correlated; but if they are distant voxels, such correlation may occur, since the

Laplacian would not have removed it. Therefore, if we plot the correlations v wt tE ε ε⎡ ⎤

⎣ ⎦ between

the innovations vtε of a fixed reference voxel v and the innovations w

tε of the set of all other

voxels, we may find large instantaneous correlation between v and certain remote voxels.

Such instantaneous spatial correlations may possibly be attributed to an intrinsic

neurophysiologic origin. The same analysis can also be applied to lagged correlations between

the innovations of pairs of voxels, i.e. v wt t lE ε ε −

⎡ ⎤⎣ ⎦ can be analyzed.

The diagram below summarizes the idea of lagged correlations between two voxels:

9

Figure 1: Schematic diagram of connectivity between two voxels.

Despite the dimensionality problem explained in the Introduction, the characteristics

of hemodynamic activities which dies out within a few seconds suggests that we

should consider a low-lag autocorrelation, which makes difficult application of

Akaike's causality to this problem. An alternative solution that helps to overcome this

limitation is by applying a cross-correlation analysis as follow.

2.3 Directed correlations between regions

Let ( vty ), 1,...,t T= , be the time series data at each voxel 1,..., vv N= and instant of

time t ; ( vtε ) 1,...,t T= , 1,...,v Nv= , the innovations resulting from fitting NN-ARx

model at each voxel v and instant of time t .

Divide the set of Nv voxels into M sets of VΩ , 1,...,V M= .

Consider an arbitrary pair of such regions, say VΩ and WΩ . For each lag

, 1,...,0,1, 2,...,l L L L= − − + , (being L the maximum lag to be considered) define ,lv wR as

the temporal correlation between ( )v

t lε +and ( )w

tε , where v ranges over the voxels in

VΩ , and w over the voxels in WΩ . Notice that ,lv wR are directed correlations, in the

sense that they represent correlations between the past at voxel v and the future at

voxel w when 0l < , and correlations between the future at voxel v and the past at

voxel w when 0l > . Thus, they could be used as descriptive statistics for exploring

possible causal relations between voxels.

Voxel-v Voxel-w

Model

v v v/ 1t t t td yξ −= −

v/ 1t ty −

w/ 1t ty −

vtd w

tdw w w

/ 1t t t td yξ −= −

( ) w v v wcorr t t t tt t tτ ξ ξ ξ ξτ= ∑ ∑ ∑−

st

tn

DATA DATA

Model

st

10

Measures of correlations between regions

The use of correlations (or multiple correlations or canonical correlations) between

past and present data at different regions is a more gross description of directed

influences between regions than those that can be achieved by a model-based

analysis. But they are a tool easy to compute and interpret.

Define ( )2

, ,l lv w v wSR R=

1. In order to summarize the correlations between two regions VΩ and WΩ at a

given lag l , a number of measurements can be considered: the mean or the median (which is a more robust estimator for the average) of the l

vwSR but also the upper 90th percentile ,

lV WUSR of the values l

vwSR over Vv∈Ω , Ww∈Ω can be

considered. (Notice that this is similar to the maximum of the values lvwSR , but this

value is less sensible to the outlayers).

2. Further summarizations over lags can be obtained by means of the measures

, ,0

11

Ll

V W V Wl

USR USRL

+

=

=+ ∑ and

0

, ,1

1l

V W V Wl L

USR USRL

−

=−

=+ ∑ , where L is the number of

lags to be taken into consideration. The fact that the sampling rate in fMRI

experiments is very high (normally about 3 seconds), it is not necessary to use

very high lag orders for the purpose of fMRI analysis. In our case, we limit this to

be 2L = . Higher lags show almost zero correlations.

3. As an alternative, a less gross summarization of the correlations over lags is obtained by considering (instead of ,v wR+ ), the multiple (temporal) correlation ,v wMR+

between the variable ( )w

tε and the set of variables ( ){ : ,..., 1,0}v

t ll Lε −= − − ; that is,

,v wMR+ is the multiple correlation between the present at voxel w and the past at

voxel v for all different lags. Similarly, ,v wMR− can also be defined as a suitable

multiple correlation.

Since ,v wMR+ for all Vv∈Ω , Ww∈Ω , are nonnegative, their upper 90th percentile,

say ,V WUMR+ , can be used to summarize the influence of the past in region VΩ on the present in region WΩ .

4. A more sophisticate measure of directed correlations between regions could be

defined by means of canonical correlation analysis. This measure is more

computer intensive and could involve difficult with singular matrices.

11

2.4 Statistical tests for Graphs

Assume we have a defined a suitable measure of dependence between two regions

VΩ and WΩ (by means of some of the proposals 1 to 4 in previous Section).

Denote such a measure by ( , )U V W . Assume that ( , )iU V W , 1,...,i Q= , are

observations of ( , )U V W in Q independent subjects.

We are interest in testing whether the actual distribution of ( , )iU V W is the same as

the distribution of ( , )U V W when there is no correlation between regions (null hypothesis), i.e., , 0l

v wR = for all Vv∈Ω , Ww∈Ω .

1) If the situation was a two sample testing problem, i.e., if we had observations ( , )iU V W , 11,...,i Q= and ( , )jU V W , 21,...,j Q= of ( , )U V W in samples of subjects

under two different conditions, then standard nonparametric tests could be used

to test whether the measure is stochastically larger in the first condition in

comparison with the second one.

But the one sample problem is much more difficult. The distribution of ( , )U V W under

the null hypothesis H is analytically unknown, even the mean value of ( , )U V W under

H is unknown. Neither standard parametric nor nonparametric tests can be applied.

2) A statistical test on the basis of simulations can be constructed even for one

subject in the following way.

Denote by VΣ and WΣ the covariance matrices of the vector of innovations

( ) ( )( : )V vVt t vε ε= ∈Ω and ( ) ( )( : )W w

Wt t wε ε= ∈Ω corresponding to the two regions

VΩ and WΩ .

Repeat for 1,...,b B= (number of simulations, 100B > ) the following steps:

a) Generate independent random vectors ( , ) ( )( : )v b vbVt t iε ε= ∈Ω and

( , ) ( )( : )w b wbWt t

wε ε= ∈Ω for 1,...,t T= , each ( , )v b

tε with distribution (0, )VN σ and each

( , )w b

tε with distribution (0, )WN σ .

b) Use ( , )v b

tε and ( , )w b

tε over Vv∈Ω , Ww∈Ω , 1,...,t T= as residuals for computing

12

the correlations ,,

l bv wR between the regions, and the corresponding measure of

dependence between regions ( , )bU V W .

Then, the upper 95% percentile 0.95VWC of the values ( , )bU V W , 1,...,b B= , can be taken

as threshold for testing the null hypothesis in one subject: the hypothesis is rejected if the observed value ( , )U V W in the subject is greater than 0.95

VWC .

In case of a sample of subjects, steps a)-b) have to be repeated also over subjects, that is for 1,...,b B= and 1,...,i Q= ; in each repetition b the measure of dependence

between regions have to be computed, say , ( , )b iU V W and also compute their

average over subjects:

,

1

1 ( , )Qb b i

VWi

U V WQU

=

= ∑

for each 1,...,b B= . Then, the upper 95% percentile 0.95VWC of the values VWT ,

1,...,b B= , is taken as threshold for testing the null hypothesis in one subject: the

hypothesis is rejected if 0.95VW

VWT C> , where VWT is the observed value of the average

of the measures over subjects in the real data

1

1 ( , )B

b

VWb

U V WBU

=

= ∑

Notice that the main computational burden of this test based on simulations is the

repeated generation of random normal variables, because other computations

involved are just correlations.

2.5 Algorithm

The diagram below summarizes the sequence of steps we follow for the data

processing:

13

Figure 2: Sequence of steps of data processing for connectivity analysis

Data Preprocessing

Before statistical modeling of the fMRI time series, the individual fMRI images were

realigned in order to remove movement-related artifacts, and the slice timing was

adjusted to that of the middle slice. Then the images were smoothed.

Being NN-ARx a dynamical model, we try to preserve the dynamic of the data as

close as possible to the original data, by applying the less possible amount of

preprocessing. We then avoid to on go into other types of preprocessing as

normalization procedures or co-registration to the anatomical image. Instead, to allow

for the possibility of doing inter-subject statistics or referring the results to a standard

brain, we prefer to normalize to the standard space provided by the Montreal

Neurological Institute (MNI), the model parameters as well as the innovations, for all

the voxels, after the model has been fitted.

NN-ARx model selection and fitting

Model selection consists in the estimation of the model parameters ( ( , , , , )p r d δΛ = Δ

(5). It is a voxelwise minimization procedure based on the AIC. The limits where each

parameter can vary are set according to the user expertise or by using some

fMRI data (Analyze format)

Data preprocessings NN-ARx AIC

Model selection

NN-ARx Model fitting

Model parameter

Innovations

Stimulus related activations plots

and statistics

Normalization

Voxelwise

Lags Connectivity

Summarized Lags Connectivity

Statistics

By Regions

Visualizations

14

common criterion. For each combination of parameters, the NN-ARx model is

calculated as well as the AIC. Then the minimum AIC is selected. For the purpose of

this work and based on our previous experience, we have fixed the neighbor's term vX to be order 1, so we avoid estimation of this parameter.

Model goodness of fit and whiteness of the innovations can be checked in a visual

way. The figure below shows the model fitting of one voxel in the calcarine sulcus for

the visual experiment. In the upper part, the fMRI signal of the voxel is shown in blue.

The one step-ahead prediction is shown in red and in green the estimated trend. Also,

optimal orders for this voxel are shown (p=1, d=2, m=1, g=2). In the middle part of

the figure, the innovations together with the conditions onsets and durations are

shown. In the lower part, some statistical tests for the innovations (Sample

Autocorrelation function, Sample Partial Autocorrelation function and the Bera-Jarque

parametric hypothesis test of composite normality) show the innovations for this voxel

are white and gaussian. This figure is very similar to Figure 5 of Riera et. al 2004b.

Figure 3: NN-ARx model fitting of a voxel in the calcarine-sulcus during the visual task: one-

step ahead prediction, trend, innovations, stimulus design and statistical tests for the

innovations are shown.

15

Global Connectivity Maps (Regions connectivity analysis)

While the voxelwise connectivity analysis will be the subject of a future work, we

concentrate our efforts here in the global connectivity analysis. For this purpose we

divide the brain into regions, as introduced in Section 2.3. There may be many

different ways for creating such regions. In our case we use the segmentation of the

gray matter of the brain provided by the MNI. In particular we use the one that divides

the brain in 71 regions (see Appendix).

Global connectivity is a gross approach and has drawbacks. One drawback is the

size of the regions, that could summarized too many different activities associated to

different brain processes in regions associated to different brain functions. The

second drawback is the method used to define the regions itself. In our case, we

standardize the images to the MNI normalized space and then we use the MNI Atlas.

But this procedure is not perfect and voxels from different brain structures can be

"confound", especially at the boundaries between regions. This could produce high

correlation values between neighbor regions without an "evident" physiological

explanation. This problem could be overcome by focused studies of connectivity

between the regions involved and successive refinements in their definitions to

smaller, better localized regions.

For the global connectivity analysis we use in this paper the measurements of connectivity defined in Section 2.3 as "lags summarization" ,V WUSR+ and ,V WUSR− . We

calculate these measurements among all the 71 regions, producing a 71 x 71

connectivity matrix for each subject. Over subjects we then apply the statistical test

described in Section 2.4.

3. Results The same sets of data used in Riera et al, 2004b were analyzed in this paper.

3.1 Visual experiment. Group analysis

Visual paradigm: A 3-T scanner (VP, General Electric, Milwaukee, WI) was used in

this experiment to collect the visual stimulus data. Ten normal volunteers (5 males

and 5 females) aged 25-43 years were used in the visual paradigm consisting of 3

blocks of 30 seconds check-board visual stimulus and 30 seconds of control

condition (starting from task condition). During the task condition, the check-board

16

was intermittently presented at a frequency of 8 Hz. Tight but comfortable foam

padding was placed around the subject’s head to minimize head movement.

fMRI parameters: Inter-scan interval TR=3 seconds. Each volume consisted of 36

slices from the bottom to the top of the head, with a voxel size of 23.44 3.44mm× in

plane, a slice thickness of 3.5 mm and a 0.5 mm gap covering the whole brain. T2-

weighted, gradient echo, echo planar imaging (EPI) sequences. (TE=30 milliseconds,

FOV=22 cm).

Parameters of scanner for anatomical reference: T2-weighted, 2D-fast spin echo

sequence (with parameters of FA=90 degree, TR=6000 milliseconds and TE=70

milliseconds) consisting of 112 trans-axial slices, with slice thickness 1.5 mm, and

pixel size was 20.859 0.859mm× .

A) B)

C)

Figure 4: 3D graph of significant correlations between brain regions for the group of subjects

under the visual task. A) Lateral, B) back, and C) top views.

17

Region 1 Region 2 -->> <<-- Region 1 Region 2 -->> <<-- LingGR LingGL 0.39166 0.37961 MidOccTmpGR ParaHipGL 0.31601 0.30634MidOccTmpGR MidOccTmpGL 0.37458 0.37135 SupParLobR CunL 0.31584 CunR CunL 0.37321 0.3662 SupParLobL PosCenGL 0.31584 LingGR MidOccTmpGR 0.37111 0.36292 LatFroOrbGR MidFroOrbGR 0.31583 0.31422PreCunL PreCunR 0.37044 0.36887 MidOccTmpGL SupTmpGL 0.3157 MidOccTmpGR CunR 0.37012 0.3617 AccumL GlobPalL 0.3157 PreCunR CunR 0.36696 0.34284 CerebL CunL 0.31566 LingGR CunR 0.36487 0.34795 PutL PutR 0.31545 MidOccTmpGR CunL 0.36323 0.35037 MidfrontOrbGL LatFroOrbGL 0.31529 0.31053LingGR CunL 0.36318 0.33961 ThalR MidOccTmpGL 0.31525 0.31051MidOccTmpGL LingGL 0.36293 0.35436 GlobPalR PutL 0.31511 0.3116 PreCunL CunL 0.36259 0.34284 PreCunL SupParLobL 0.31484 0.31307MidOccTmpGL CunL 0.3593 0.34752 ThalL BrStem 0.31484 ThalL ThalR 0.35696 0.34913 PreCunR SupOccGR 0.31466 MidOccTmpGR LingGL 0.35634 0.34667 ThalR CerebR 0.31457 0.313 LingGL CunL 0.35561 0.34742 SupOccGL PreCunR 0.31449 0.31364LingGR MidOccTmpGL 0.355 0.34932 SupParLobL CunL 0.31449 MidOccTmpGL CunR 0.35103 0.34176 PosCenGL PreCenGR 0.31449 0.30646PreCunR CunL 0.34921 0.32169 ThalR BrStem 0.31448 PreCunL CunR 0.34828 0.33071 MidOccTmpGR SupTmpGL 0.31432 CunR LingGL 0.34686 0.34549 CaudNucR PutR 0.31431 ParaHipGR MidOccTmpGR 0.34657 0.3337 MidOccGL SupOccGL 0.31409 0.31329LingGR LatOccTmpGR 0.34639 0.33674 MidOccTmpGR SupTmpGR 0.31397 MidFroOrbGR MidfrontOrbGL 0.34494 0.34474 CerebR CunL 0.31389 ThalL SubThaNucL 0.34485 0.33185 ParaHipGR LingGR 0.31373 0.31019SupOccGR CunR 0.33906 0.32841 ThalL ParaHipGL 0.31373 0.30962OccpoleR LingGR 0.33819 0.33163 SupParLobR CunR 0.31358 MidOccTmpGR LatOccTmpGR 0.33802 0.33226 PosCenGR PreCenGR 0.31346 0.31316SupOccGL CunL 0.33584 0.31798 ParaHipGR ThalR 0.3134 0.31093SupOccGR CunL 0.3335 0.32282 ParaHipGR PreCunR 0.31308 0.31275OccpoleR InfOccGR 0.33298 0.31707 SupParLobR PosCenGL 0.31295 GlobPalR PutR 0.33282 0.3139 OccpoleL OccpoleR 0.31275 0.31122SupOccGR SupOccGL 0.33275 0.33206 ThalL SupTmpGL 0.3127 PreCunL SupOccGL 0.33274 0.32895 ThalR InsL 0.3126 LingGL LatOccTmpGR 0.33136 0.32905 LatFroOrbGR MidfrontOrbGL 0.31258 0.31256CerebL LingGL 0.33068 0.32445 SupOccGR MidOccTmpGL 0.31258 0.31024LingGR CerebL 0.33052 0.32687 HipoCL UncusL 0.31241 SupOccGL CunR 0.33028 0.31366 OccpoleR LatOccTmpGR 0.31239 ThalL MidOccTmpGL 0.32976 0.32128 SupTmpGR SupTmpGL 0.31238 0.30968ThalL LingGL 0.32843 0.3099 UncusL UncusR 0.31237 0.30992SupParLobR SupOccGR 0.32837 0.32576 PosCenGR PreCenGL 0.31201 0.30898LingGR CerebR 0.32828 0.32226 CerebL CunR 0.31198 ThalR GlobPalR 0.32824 0.31367 LingGR PreCunL 0.31167 OccpoleR LingGL 0.32822 0.31643 LingGR InfOccGR 0.31162 0.30695MidOccTmpGL LatOccTmpGR 0.32777 0.32718 SupOccGL MidOccTmpGR 0.31147 ParaHipGR MidOccTmpGL 0.32768 0.32279 ThalL GlobPalR 0.31135 ThalL LingGR 0.32754 0.32236 HipoCL ThalL 0.31132 0.31041ThalL CunL 0.32752 0.30745 PreCunR CingRegL 0.3113 ThalR LingGR 0.32712 0.3269 BrStem UncusR 0.31095 CaudNucR ThalR 0.32631 0.32522 ParaHipGR BrStem 0.31092 ThalL MidOccTmpGR 0.32627 0.32029 ThalL AccumR 0.31062 OccpoleL LingGL 0.32603 0.32108 PreCunL SupParLobR 0.3106 0.30795MidOccTmpGR SupOccGR 0.32531 0.32485 ParaHipGR UncusL 0.31056 OccpoleR SupOccGR 0.32487 0.31103 MidOccTmpGR InsR 0.31053 CerebL MidOccTmpGL 0.32431 0.31659 MidOccGL PreCunL 0.31051 0.30955ThalR MidOccTmpGR 0.32406 0.31296 ThalL SupTmpGR 0.31044 CerebR MidOccTmpGR 0.32402 0.32009 ParaHipGR LingGL 0.31031 ThalR LingGL 0.32398 0.31043 ThalL LatOccTmpGR 0.31021 MidOccTmpGL ParaHipGL 0.32383 0.31187 LingGR ParaHipGL 0.31017 OccpoleL LingGR 0.32377 0.31518 MidOccTmpGL HipoCL 0.31015 ParaHipGR ThalL 0.32373 0.31127 InsL SupTmpGL 0.31009 CerebL MidOccTmpGR 0.32366 0.323 CerebL LatOccTmpGR 0.31007 SubThaNucR ThalR 0.32342 0.31744 BrStem UncusL 0.30993

18

PosCenGL PosCenGR 0.32327 0.31365 CerebR CunR 0.3098 ThalR PutR 0.3231 SubThaNucR BrStem 0.3098 ThalL PutR 0.32255 SupOccGL MidOccTmpGL 0.30974 CaudNucR PutL 0.32239 CerebR LatOccTmpGR 0.30961 ThalL CaudNucL 0.32234 0.32116 SupParLobL SupOccGR 0.30958 0.30954PreCunL MidOccTmpGL 0.32209 0.31515 PreCunL LingGL 0.30955 PosCenGL PreCenGL 0.32179 0.3119 ThalR CunR 0.30949 ParaHipGR CunR 0.32173 0.30725 SupParLobR PreCenGL 0.30921 ThalL PutL 0.32138 MidOccTmpGR HipoCL 0.30903 PreCunR MidOccTmpGR 0.32113 0.31356 SupParLobR MidOccTmpGR 0.309 LatOccTmpGR CunL 0.32089 0.30842 ParaHipGR LatOccTmpGR 0.30897 ThalL CingRegL 0.32081 ThalL PreCunL 0.30891 0.30756CaudNucR AccumR 0.32066 0.30689 CaudNucL ThalR 0.30874 ParaHipGR CunL 0.32051 OccpoleL InfOccGL 0.30864 ThalL CerebR 0.3205 0.31803 PreCunR SupParLobR 0.30852 0.30805LingGR SupOccGR 0.32041 0.31315 OccpoleL SupOccGR 0.30852 GlobPalR AccumR 0.3204 0.31221 UncusL AccumL 0.30842 0.30727ThalL InsR 0.32019 SupOccGL LingGL 0.30832 ThalR PutL 0.32005 PreCunR MidOccTmpGL 0.30815 PutL InsL 0.32001 0.31211 CaudNucL PutL 0.30803 CerebR LingGL 0.31952 0.31148 LingGR PutR 0.30788 PreCunL CingRegL 0.31949 ThalR InsR 0.30786 CerebR MidOccTmpGL 0.31937 0.31335 LingGR InsL 0.30782 CerebR CerebL 0.31924 0.31683 SupTmpGR SupMarGR 0.3077 MidOccGR SupOccGR 0.31885 0.31634 ThalL SupOccGR 0.3077 ThalL CerebL 0.31856 0.31481 PreCunL ParaHipGL 0.30745 SupParLobL SupParLobR 0.31849 0.31806 CingR CingRegL 0.30744 SupParLobR PosCenGR 0.31845 SupParLobL CunR 0.3074 OccpoleR CunR 0.31837 SupTmpGR InsR 0.30717 ThalL CunR 0.3176 ThalR LatOccTmpGR 0.30711 SupOccGR LingGL 0.31752 0.31246 CunL CingRegL 0.30711 ParaHipGR ParaHipGL 0.31741 0.31035 MidFroOrbGR LatFroOrbGL 0.30708 ThalR CerebL 0.31731 0.31005 SupParLobL AnguGL 0.30687 ThalL CaudNucR 0.31728 0.31187 SupParLobR PreCenGR 0.30674 SupParLobL SupOccGL 0.31728 0.31714 OccpoleR SupOccGL 0.30667 CaudNucR GlobPalR 0.31726 0.30801 SupParLobL PreCenGL 0.30666 SupOccGL SupParLobR 0.31716 0.31172 LingGR HipoCL 0.30663 LatOccTmpGR CunR 0.31713 0.31187 PosCenGR SupMarGR 0.30662 ThalR CunL 0.31701 PreCunL ParaHipGR 0.3066 PreCunL MidOccTmpGR 0.317 0.31326 ParaHipGR UncusR 0.30656 ThalL GlobPalL 0.31676 LingGR SupOccGL 0.30652 ThalR SubThaNucL 0.31672 0.30781 ParaHipGR CerebR 0.30651 MidOccTmpGL SupTmpGR 0.31648 LingGR SupTmpGR 0.3064 OccpoleL SupOccGL 0.31638 0.30686 OccpoleR CunL 0.31622 ThalL InsL 0.31617

Table 1: Average value of the significant correlations between the brain regions for the group

of subjects under the visual task. Sorted in descending order.

19

Figure 5: Map of significant correlations for the group of subjects under the visual task. B

lue triangles inside the rectangle mean connectivity

direction going from the blue to the red region. Vice versa, red triangles indicate the direction of the correlation from

red to blue regions.

20

3.2 Motor experiment. Group analysis

Motor paradigm: A 1.5-T scanner (Vision, Siemens Erlangen, Germany) was used in

this experiment to collect the data. Five right-handed, normal volunteers (3 male and

2 females) aged 24-37 years were used in the motor paradigm consisting of 9 blocks

of 60 Secs moving conditions and a 60 Secs resting condition(starting frm resting

condition). The subjects were asked to perform right hand movement tasks. During

the moving condition, a small circle at the center of the screen was used as a cue

(lasting for 200 mSec) indicating the subject should close its hand and a cross

indicating to open it. Each subject’s head was fixed using ear fixation blocks.

fMRI parameters: Inter-scan interval TR=1.2 Secs. Each volume consisted of 8 slices

from top to bottom of the head, with a voxel size of 3 x 3 mm in plane, a slice

thickness of 10 mm and with a 5 mm gap covering the whole brain. T2-weighted,

gradient-echo, echo-planer imaging (EPI) sequences (TE=60 mSecs, FA=90

degrees).

Parameters of scanner for anatomical reference: Spoiled gradient-echo sequence

(recovery time TR=9.7 mSecs, echo time TE=4 mSecs, FA=12 degrees) consisting of

96 slices with a voxel size of 1.25 x 0.9 x 1.92 mm.

A) B)

21

C)

Figure 6: 3D graph of significant correlations between brain regions for the group of subjects

under the motor task: A) Lateral, B) back, and C) top views.

22

Region 1 Region 2 -->> <<-- Region 1 Region 2 -->> <<-- ThalL SubThaNucL 0.31673 0.29609 GlobPalR HipoCR 0.21428 0.21052ThalR SubThaNucR 0.30127 0.28517 PutR UncusL 0.21397 0.19843GlobPalL AccumL 0.28733 0.28492 SubThaNucR GlobPalL 0.21323 AccumR GlobPalR 0.28183 0.2754 LingGR CunR 0.21315 0.20676PutR GlobPalR 0.27908 0.27871 LatFroOrbGR AccumR 0.21306 0.21172HipoCL UncusL 0.27714 0.25749 ParaHipGR BrStem 0.21256 0.21026CaudNucR AccumR 0.27703 0.26357 CaudNucL ThalL 0.21228 0.20515HipoCL ParaHipGL 0.27307 0.26639 PutL GlobPalR 0.21194 0.21189GlobPalL PutL 0.27265 0.26913 LingGR MidOccTmpGR 0.21186 0.20967UncusL UncusR 0.271 0.26831 ThalL PutL 0.21156 SubThaNucL GlobPalL 0.26943 0.24097 CaudNucR UncusR 0.21144 CaudNucL AccumL 0.2691 0.25064 PosCenGR PreCenGR 0.21094 0.21076ParaHipGL UncusL 0.26316 0.25337 AccumL UncusR 0.21077 PreCunR PreCunL 0.26299 0.26071 GlobPalL PutR 0.21043 0.20768GlobPalL UncusL 0.25995 0.23933 ThalL ParaHipGR 0.20973 BrStem UncusL 0.25801 0.24699 PutL AccumR 0.20956 0.20697HipoCL UncusR 0.25719 0.24031 MidFroGL MidFroGR 0.2094 0.20775SubThaNucL UncusL 0.25355 0.21428 SubThaNucL HipoCR 0.20934 HipoCR UncusR 0.25353 0.24741 MidOccTmpGL CunR 0.20914 GlobPalR UncusR 0.25145 0.23626 PutL UncusR 0.20835 0.20094SubThaNucR UncusR 0.25027 0.21963 PutR PutL 0.20829 0.2037 ThalL ThalR 0.2495 0.24146 PutR HipoCR 0.20798 0.20187ThalL SubThaNucR 0.24852 0.22416 CunL LingGL 0.20776 0.20532SubThaNucR BrStem 0.2466 0.23751 MidOccTmpGL LingGR 0.20702 0.1975 ParaHipGR HipoCR 0.24441 0.24287 ThalR PutL 0.20696 BrStem UncusR 0.24433 0.23567 SubThaNucR PutR 0.20671 0.19708HipoCL BrStem 0.24316 0.23654 ThalL AccumR 0.20668 ThalL UncusL 0.24311 0.20342 ThalL HipoCR 0.20621 AccumR PutR 0.24301 0.23784 CaudNucL UncusR 0.20607 PreCunR CunR 0.24298 0.2325 MidOccTmpGL HipoCL 0.2056 ParaHipGL UncusR 0.23924 0.23027 MidTmpGL UncusL 0.20558 HipoCR UncusL 0.23879 0.23339 CaudNucL AccumR 0.20556 SubThaNucL BrStem 0.23869 0.22359 ParaHipGR MidOccTmpGR 0.20545 0.20295ThalL GlobPalL 0.23708 0.19894 GlobPalL HipoCL 0.2052 0.19933ParaHipGR UncusR 0.23612 0.22403 CaudNucR GlobPalL 0.20507 0.1969 AccumL UncusL 0.23609 0.22937 SupTmpGL UncusR 0.20505 HipoCL HipoCR 0.23558 0.2309 MidOccTmpGR CunL 0.20494 0.20344SubThaNucL UncusR 0.23548 0.20177 ThalR PutR 0.20479 SubThaNucR UncusL 0.23539 0.21129 PreCunR ParaHipGR 0.20463 0.20054ParaHipGL BrStem 0.2353 0.23519 PreCunR CunL 0.20456 0.19783PutL UncusL 0.23485 0.21956 CaudNucL PutL 0.20429 PutL AccumL 0.23388 0.2262 SupTmpGR UncusR 0.20429 ThalR SubThaNucL 0.23325 0.22663 CaudNucL GlobPalL 0.20397 CunL CunR 0.23292 0.22806 ThalR HipoCR 0.20383 LingGR LingGL 0.23216 0.22843 SubThaNucL SupTmpGL 0.20378 SubThaNucL SubThaNucR 0.2321 0.20569 GlobPalL HipoCR 0.20346 AccumR UncusR 0.23143 0.2123 SupParLobR SupParLobL 0.2033 0.19674ParaHipGR UncusL 0.22911 0.2189 InsR UncusL 0.2033 MidOccTmpGL MidOccTmpGR 0.22837 0.22288 HipoCL SupTmpGL 0.20329 0.19982GlobPalL UncusR 0.22823 0.20719 MidOccTmpGL UncusL 0.20313 PreCunL CunL 0.22817 0.21788 ThalR CaudNucR 0.20312 0.19908MidfrontOrbGL MidFroOrbGR 0.22754 0.22655 SupOccGL CunL 0.20305 GlobPalR UncusL 0.22741 0.21883 InsR SupTmpGR 0.20298 0.20264ThalL UncusR 0.2262 InsR InsL 0.20296 0.20248CaudNucL UncusL 0.22588 CaudNucR AccumL 0.20277 SubThaNucR HipoCL 0.22571 0.2217 SubThaNucR SupTmpGL 0.2026 CaudNucR GlobPalR 0.2255 0.2117 MidOccTmpGL CerebL 0.20259 ThalL HipoCL 0.225 0.20261 ThalR AccumR 0.20256 GlobPalL GlobPalR 0.22486 0.22245 SupTmpGR SupMarGR 0.20246 0.20238PutR InsR 0.2248 0.22413 ThalL AccumL 0.20181 SubThaNucR HipoCR 0.22428 0.20637 SubThaNucR InsL 0.20141 AccumR UncusL 0.2238 0.20743 SubThaNucL ParaHipGR 0.20117 PutR UncusR 0.22348 0.20904 LatOccTmpGL HipoCL 0.20103 0.19932

23

MidOccTmpGR CunR 0.22284 0.21441 LatFroOrbGR UncusR 0.20057 CaudNucR CaudNucL 0.22283 0.22223 HipoCL PutL 0.20049 0.1988 ThalL BrStem 0.22272 0.20686 SubThaNucL PutR 0.20049 HipoCL ParaHipGR 0.22261 0.21909 MidOccTmpGR LingGL 0.20048 0.19925ThalR GlobPalR 0.22236 0.19993 PosCenGL PreCenGL 0.20036 0.19943SubThaNucR PutL 0.22163 0.19861 ThalR HipoCL 0.20018 SubThaNucL HipoCL 0.22158 0.20496 InsL SupTmpGL 0.20014 0.19893InsL PutL 0.2208 0.21922 CerebR LingGR 0.2 ThalR UncusL 0.22064 ThalR ParaHipGR 0.19993 MidOccTmpGL LingGL 0.22048 0.2103 SubThaNucR SupTmpGR 0.19982 ThalR GlobPalL 0.22037 MidTmpGR UncusL 0.19933 CingRegL CingR 0.22025 0.21636 AccumL GlobPalR 0.19931 SubThaNucL ParaHipGL 0.22024 0.202 PosCenGR SupMarGR 0.1992 0.19894SubThaNucR GlobPalR 0.22016 ThalL PutR 0.19897 MidOccTmpGL CunL 0.21989 0.21255 InsR UncusR 0.19893 AccumR AccumL 0.21987 0.21915 PreCunL MidOccTmpGL 0.19893 GlobPalL AccumR 0.21923 0.21829 HipoCR InsR 0.19885 SupTmpGL UncusL 0.21841 0.20417 SubThaNucL SupTmpGR 0.19876 SubThaNucL GlobPalR 0.21834 0.20205 SupOccGR SupOccGL 0.19875 ParaHipGR ParaHipGL 0.21797 0.21739 CerebR CerebL 0.19845 ThalL ParaHipGL 0.21787 0.20029 SupOccGR CunR 0.1982 SubThaNucL PutL 0.21769 LatFroOrbGR UncusL 0.19811 SubThaNucR ParaHipGR 0.21749 0.21423 PreCunR SubThaNucR 0.19799 SupTmpGR UncusL 0.21726 0.19776 SubThaNucR InsR 0.19765 MidOccTmpGL ParaHipGL 0.2166 0.20551 CaudNucR PutL 0.19727 InsL UncusL 0.2165 0.20262 PutR InsL 0.19723 0.19719CaudNucR UncusL 0.21615 HipoCL GlobPalR 0.19721 SubThaNucR ParaHipGL 0.21529 0.21036 MidOccTmpGL CerebR 0.19719 CaudNucR PutR 0.21501 0.2098 SupTmpGR HipoCR 0.19717 HipoCR ParaHipGL 0.21493 0.21395 InsL UncusR 0.19696 BrStem HipoCR 0.21477 0.21323 SupParLobR AnguGR 0.19673 ThalL GlobPalR 0.21451 PreCunL CunR 0.2145 0.20109 ThalR UncusR 0.21439

Table 2: Average value of the significant correlations between the brain regions for the group

of subjects under the motor task. Sorted in descending order.

24

Figure 7: Map of significant correlations for the group of subjects under the m

otor task. Blue triangles inside the rectangle m

ean connectivity

direction going from the blue to the red region. Vice versa, red triangle indicate the direction of the correlation from

red to blue regions.

25

3.3 Brain connectivity base on NN-ARx in other experimental

situations. What the method can do.

A) B)

C)

Figure 8: 3D graph of significant correlations between brain regions of a blind subject during a

tactile discrimination task. A single case analysis: A) Left, B) back, and C) top views.

26

4. Discussion

4.1 Visual pathway

4.2 Motor pathway

4.3 Connectivity associated to tactile discrimination in blind subject.

A large correlation between two regions or remote voxels may imply a kind of fast

neural connectivity between them. The correlations may result from either functional

connectivity or effective connectivity. However, since the fMRI data is recorded with a

sampling time of a few seconds, it is inappropriate to judge whether this connectivity

was functional or effective. To give a reasonable answer to this question, study of the

fusion of simultaneously recorded fMRI and EEG time series (see Riera et al., 2005)

will be required.

4.4 Dynamical modeling of fMRI time series

We would like to point out that the methodology based on the innovation approach,

as discussed in this paper, is closely related to other recent studies on the subject of

connectivity in the context of neuroscience (Bernasconi and Koenig, 1999; Kaminski

et al., 2001; Baccala and Sameshima, 2001; Goebel et al., 2003; Harrison et al.,

2003; Yamashita et al., 2005). Specifically, the NN-ARx model used in the present

work, represents the case of extraordinarily large dimension and sparse parameter

matrices, therefore it is compatible with the methodology developed by Akaike (1968),

Granger (1969) and Geweke (1982) on the basis of AR modeling. It would be an

interesting topic for future research to apply this methodology to full-dimensional AR

models derived from NN-ARx models of fMRI time series from various cognitive or

sensorimotor experiments.

Now we would like to discuss certain points which are related to the two prerequisite

conditions of our method for fMRI data analysis, i.e. the issue of estimating fast

neural dynamics from the slow hemodynamic BOLD signals; and the issue of using

time series data of extraordinarily large dimension. The first issue inevitably leads us

to the notoriously difficult problem of “aliasing”. As mentioned briefly in the

27

introduction, it is, in general, impossible, to estimate spectral properties on a

millisecond resolution from time series sampled every few seconds. However, if

additional information and constraints can be imported, it may become feasible. This

case is assumed in Friston et al. (2003), where the fast neural dynamics is chosen to

be deterministic and a relation between the neural mass model and the slow

hemodynamic model is imposed. It seems finding a realistic relation between the

neural dynamics and the hemodynamics is a critical and important problem for future

work on connectivity. There are also no problems related to aliasing when there is no

important information in the fast dynamics. This is the case for the successful

simulation studies of Goebel et al. (2003). Their model, however, does not

correspond to realistic EEG dynamics, since the main frequencies of the EEG, as

characterized by their model, is lower than 1 Hz.

On the other hand, fMRI time series, far from being completely random, show

obvious patterns of temporal and spatial correlation structure. A plausible

physiological interpretation of instantaneous correlations between distant voxels

could be that spatio-temporal hemodynamic activities triggered by fast neural

dynamical seem to occur almost simultaneously, when recorded by a low sampling

frequency. But we also find lagged correlations in fMRI time series, e.g. over a lag of

one sampling period. It will be very difficult to provide meaningful interpretations for

such correlations, unless additional physiological information relating to the specific

experimental setup (e.g. cognitive task, sensory stimulation, etc.) can be taken into

account. Notwithstanding this caveat there is by now firm statistical evidence for slow

spatio-temporal hemodynamic processes which might originally be triggered by fast

neural dynamical activity occurring in brain during the experiments.

It is surprising that in most statistical studies of causal relationships much effort has

been expended on the case that dynamical correlations and instantaneous

correlations are present at the same time, such that the noise covariance matrix of

corresponding autoregressive models is non-diagonal. For analyzing brain electrical

activity gathered from intracranial recordings (Bernasconi and Koenig, 1999; Brovelli

et al., 2004) and fMRI time series (Goebel et al., 2003; Yamashita et al., 2005) under

this assumption elaborate quantitative studies of Wiener-Granger causality have

been discussed. However, it must be remembered that the genuine Wiener-Granger

causality can be meaningful only if the set of available information, in the present

case consisting of the past and the present of the time series, contains all possibly

28

relevant information for the problem. Bernasconi and Koenig (1999) correctly stress

this point by writing “From the conceptual and practical point of view, the most

delicate issue concerns the assumption of the completeness of the information set

we use to make statements about the causality in our system […] in this sense, it is a

clear advantage to be able to attack the problem from the multivariate point of view,

by modeling multiple channels at one time”. Even more, Akaike (1968; 1972), after

introducing the method of dynamical causality analysis in frequency-domain,

recommended to reconsider the design of the sampling of the data and to look for the

possibility of using more measurements, i.e. increasing the dimension of the data, in

order to analyze whether the off-diagonal elements of the noise covariance matrix are

sufficiently small to be ignored.

It has been shown that there exists some nonlinear dynamical structure in BOLD

signals (Friston et al., 2000; Riera et al., 2004a). On the other hand, Lahaye et al.

(2003) report that nonlinearity in fMRI time series is only weak. Our simulation study

also implies that linear modeling approaches are quite robust in detecting

instantaneous correlations, even though the time series was generated by a

nonlinear system. However, this does not mean that the ability of detecting

instantaneous correlations could not be improved further by employing nonlinear

dynamical models. The generalization of the NN-ARx model methodology into the

realm of nonlinear models, e.g. by introducing an instantaneously applied sigmoid

function or by generalizing the AR parameters to become state-dependent (Priestley,

1988; Ozaki et al., 2004) may be an interesting topic for future research.

4.5 Causal Modeling and the Innovation Approach

It is a basic concept of time series analysis that, upon observing temporal

correlations in the data, the presence of underlying dynamical processes producing

these correlations is inferred. In the physical world, such as the human brain in vivo,

future activity never affects and changes the past. Temporal dependencies are

always directed from the past to the future. In this sense we could speak of the model

of the dynamics as a causal model. A useful way to find a causal model from time

series data is to consider the prediction of the series. We aim at removing temporal dependencies on the past can and to whiten the observed time series tx into an

independent series by subtracting the predicted value from the realized (and observed) data value tx :

29

1 2| , ,...t t t tt x E x x xε − −⎡ ⎤= − ⎣ ⎦

To perform this whitening step we need to find a suitable dynamical model providing

the prediction of the time series by using past observations. Whitening, i.e. removing

the temporal dependencies on the past, and finding a suitable causal dynamical

model is in fact the same thing. By whitening the causal relations are extracted from

the observed time series, and a relevant dynamical model is obtained.

This concept has lead to the well-known innovation approach developed by N.

Wiener in the 1930s. We expect that the dynamical process is always exposed to

unpredictable noise inputs, as well as organized stimuli; both noise and stimulus

inputs accumulate in the dynamics and thereby affect the future. In this framework

the prediction errors (i.e., the innovations) are not a nuisance, but a useful source of

information that contains the key to explaining the dynamical processes in which we

are interested.

The concepts described so far are supported by a very strong mathematical theorem

given by Levy (1956; see also theorem 41 in Protter, 1990) which states that “for any continuous–time Markov process tx the corresponding innovations can be

represented, under mild conditions, as the sum of two white noise processes, namely

a Gaussian noise process and a Poisson noise process”. This theorem is a stronger

version of the well-known theorem for Markov diffusion processes (Ito, 1951; Doob,

1953), according to which “any dynamical process can be represented by a

differential equation driven by Gaussian white noise, if the process is Markov and

continuous (i.e. without any discontinuous jump)”. The case of additional observation

noise has been treated by Frost and Kailath (1971). Consequently we expect that,

under the assumption of continuous dynamics, the time series of resulting

innovations, for an optimal predictor, will be uncorrelated (in fact, independent) and

Gaussian, even if, due to possible nonlinearities in dynamics, the process is non-

Gaussian distributed. This theorem implies that, if we employ a properly chosen

model for the dynamics, the prediction errors will be distributed as Gaussian white

noise. Then the log-likelihood function for the time series may be calculated using the

standard Gaussian likelihood, even though the original observed time series may

have displayed nonlinearities and non-Gaussian distribution.

The methodology based on Levy’s theorem has been employed in time series

30

analysis since the early 1990s (Ozaki, 1992; Ozaki and Iino, 2000; Ozaki et al.,

2004); in the neurosciences it has been applied to the identification of dynamical

causal models, such as the Zetterberg model for EEG time series (Valdes et al.,

1999) and Balloon model for fMRI time series (Riera et al., 2004a). In the general

framework of Frost and Kailath (1971), the two types of noise, dynamical noise and

observation noise, play different roles: dynamical noise affects the present and the

future of the process, while observation noise never affects the future of the process.

Through nonlinear filtering, combined with the maximum log-likelihood approach, it is

possible to identify the strengths of their respective contributions to the observed time

series. By this methodology it becomes also possible to solve signal decomposition

problems like the “Cocktail Party Problem”, well-known in Signal Processing. Here

the important point is that the causal innovation approach is capable of processing

and identifying details relating to the temporal ordering of the speech, i.e. the signal,

and the noise in the party more carefully than non-causal approaches of time series

analysis, such as Independent Component Analysis (ICA).

It is a notoriously difficult problem in the practical application of the innovation

approach that there could be many possible candidates for causal models, and each

causal model will whiten the series in a different way and give rise to different

prediction errors. Also Wiener-Granger Causality is not free from this model

identification problem. It seems natural to choose among several candidate models

the model that produces the minimum prediction error variance. But clearly it is

necessary to be careful about the risk of over fitting, resulting from the use of a too

large number of model parameters. For this situation it has been recommended,

especially in the case of the study of real-world data (as opposed to artificially

generated data) to employ the Akaike Information Criterion, AIC (Akaike, 1985;

Shibata, 1980).

4.6 Limitations

This methodology has not been tested for the case of multi-stimulus or even-related

designs as well as for too short data set experiments.

Forthcoming work 1. Connectivity maps and statistics for independently task and resting conditions

separately, as well as for each condition separately in the case of multi-

31

stimulus design.

2. Connectivity maps and statistics for comparing data set with different

paradigms design or, for example, experiment with condition versus only plain

resting period recording.

3. Connectivity analysis voxel-wise oriented as well as user-defined regions.

4. Separate analysis for negative and positive correlations.

Acknowledgements T. Ozaki gratefully acknowledges support by the Japanese Society for the Promotion

of Science (JSPS) through grant KIBAN-(B) 173000922301.

A. Galka gratefully acknowledges support by the Deutsche Forschungsgemeinschaft

(DFG) through project GA 673/1-1 and by JSPS through fellowship ID No. P 03059.

32

Appendix

71 regions gray matter segmentation defined by the MNI

1=Mid-FrontoOrb-Gy-R 2=Mid-Front-Gy-R 3=Insula-R 4=Precentral-Gy-R 5=Lat-FrontoOrb-Gy-R 6=Cingulate-R 7=Mid-Front-Gy-L 8=Sup-Front-Gy-R 9=Globus-pallidus-R 10=Globus-pallidus-L 11=Putamen-L 12=Inf-Front-Gy-L 13=Putamen-R 14=Parahippocampal-Gy-L 15=Angular-Gy-R 16=Brain-stem 17=Subthalamic-Nuc-R 18=Accumbens-R 19=Uncus-R 20=Cingulate-region-L 21=Precuneus-R 22=Subthalamic-Nuc-L 23=Hippocampal-R 24=Inf-Occip-Gy-L 25=Sup-Occip-Gy-R 26=Caudate-Nuc-L 27=Supramarginal-Gy-L 28=Mid-Front-Gy-L 29=Sup-Pariet-Lob-L 30=Caudate-Nuc-R 31=Cuneus-L 32=Precuneus-L 33=Supramarginal-Gy-R 34=Sup-Temp-Gy-L 35=Uncus-L 36=Mid-Occip-Gy-R

37=Mid-Temp-Gy-L 38=Cerebellum-L 39=Lingual-Gy-L 40=Sup-Front-Gy-L 41=Accumbens-L 42=Postcentral-Gy-L 43=Inf-Front-Gy-R 44=Cerebellum-R 45=Precentral-Gy-L 46=Mid-front-Orb-Gy-L 47=Sup-Pariet-Lob-R 48=Lat-Front-Orb-Gy-L 49=Inf-Occip-Gy-R 50=Sup-Occip-Gy-L 51=Lat-OccipTemp-Gy-R 52=Hippocampal-L 53=Thalamus-L 54=Insula-L 55=Postcentral-Gy-R 56=Lingual-Gy-R 57=Mid-Front-Gy-R 58=Mid-OccipTemp-Gy-L 59=Parahippocampal-Gy-R 60=Mid-Temp-Gy-R 61=Occip-pole-R 62=Inf-Temp-Gy-R 63=Sup-Temp-Gy-R 64=Mid-Occip-Gy-L 65=Angular-Gy-L 66=Inf-Temp-Gy-L 67=Mid-OccipTemp-Gy-R 68=Cuneus-R 69=Lat-OccipTemp-Gy-L 70=Thalamus-R 71=Occip-pole-L

33

References - Akaike, H. (1968), On the use of a linear model for the identification of

feedback systems, Ann. Inst. Stat. Math. 20, 425-439.

- Akaike, H. (1969), Fitting autoregressive models for prediction, Ann. Inst. Stat.

Math. 21, 243-247.

- Akaike, H. (1972), Statistical analysis and Control of Dynamical Systems,

Tokyo: Saiensusha, (in Japanese); English translation available from Kluwer

Academic Publishers (1988).

- Akaike, H. (1973), Information theory and an extension of maximum likelihood

principle, in: Proceeding of 2nd International Symposium on Information

Theory, B.N. Petrov and F. Csaki, Eds., Akademiai-Kiado, Budapest, 267-281.

- Akaike, H. (1985), Prediction and Entropy, in A Celebration of Statistics, A.C.

Atkinson and S.E. Feinberg, Eds., Springer, Berlin, Heidelberg, New York, 1-

24.

- Baccala, L.A., and Sameshima, K. (2001), Partial directed coherence: a new

concept in neural structure determination, Biol. Cyb. 84, 463-474.

- Bernasconi, C., and Koenig, P. (1999), On the directionality of cortical

interactions studied by structural analysis of electrophysiological recordings,

Biol. Cyb. 81, 199-210.

- Boltzmann, L. (1877), Ueber die Beziehung zwischen dem zweiten Hauptsatze

der mechanischen Waermetheorie und der Wahrscheinlichkeitsrechnung

respective den Saetzen ueber das Waermegleichgewicht, Wiener Berichte 76,

373-435.

- Brillinger, D.R. (2002), Second-order moments and mutual information in the

analysis of time series, in: Recent Advances in Statistical Methods, Y.P.

Chaubey, Ed., Imperial College Press, London, 64-76.

- Brovelli, A., Ding, M., Ledberg, A., Chen, Y., Nakamura, R., and Bressler, L.

(2004), Beta oscillator in a large-scale sensorimotor cortical network:

Directional influences revealed by Granger Causality, Proc. Nat. Acad. Sci.

101, 9849-9854.

- Buechel, C., and Friston, K.J. (1997), Modulation of connectivity in visual

34

pathways by attention: cortical interactions evaluated with structural equation

modeling and fMRI, Cereb. Cortex 7, 768-778.

- David, O., Cosmelli, D., and Friston, K.J. (2004), Evaluation of different

measures of functional connectivity using a neural mass model, Neuroimage

21, 659-673.

- Dodel, S., Hellmann, J.M., and Geisel, T. (2002), Functional connectivity by

cross-correlation clustering, Neurocomputing 44-46, 1065-1077.

- Doob, J.L. (1953), Stochastic Processes, Wiley, New York.

- Friston, K.J., Mechelli, A., Turner, R., and Price, C.J. (2000), Nonlinear

responses in fMRI: the balloon model, Volterra kernels and other

hemodynamics, Neuroimage 12, 466-477.

− -Friston, K.J., Harrison, L., and Penny, W. (2003), Dynamic Causal Modeling, Neuroimage 19, 1273-1302.

- Frost, P.A., and Kailath, T. (1971), An innovation approach to least squares

estimation – part III: Nonlinear estimation in white Gaussian noise, IEEE Trans.

Autom. Contr. 16, 217-226.

- Galka, A., Yamashita, O., Ozaki, T., Biscay, R., and Valdes-Sosa, P. (2004), A

solution to the dynamical inverse problem of EEG generation using

spatiotemporal Kalman filtering, Neuroimage 23, 435–453.

- Goebel, R., Roebroeck, A., Kim, D., and Formisano, E. (2003), Investigating

directed cortical interactions in time-resolved fMRI data using vector

autoregressive modeling and Granger causality mapping, Magnetic

Resonance Imaging 21, 1251-1261.

- Granger, C.W.J. (1969), Investigating causal relations by econometric models

and cross-spectral methods, Econometrica 37, 424-438.

- Geweke, J. (1982), Measurement of linear dependence and feedback

between multiple time series, J. Am. Stat. Assoc. 77, 304-313.

- Haggan, V., and Ozaki, T. (1981), Modelling nonlinear random vibrations using

an amplitude-dependent autoregressive time series model, Biometrika 68,

189-196.

- Hall, P., and Morton, S.C. (1993), On the estimation of entropy, Ann. Inst. Stat.

35

Math. 45, 69-88.

- Harrison, L., Penny, W.D., and Friston, K.J. (2003), Multivariate autoregressive

modeling of fMRI time series, Neuroimage 19, 1477-1491.

- Ioannides, A., Liu, L.C., Kwapien, J., Drozdz, S., and Streit, M. (2000),

Coupling of regional activations in a human brain during an object and face

recognition task, Human Brain Mapping 11, 77-92.

- Ito, K. (1951), On stochastic differential equations, Mem. Am. Math. Soc. 4, 1-

51.

- Kailath, T. (1974), A view of three decades of linear filtering theory, IEEE Trans.

Inform. Theory 20, 145-181.

- Kaneko, K. (1989), Spatiotemporal chaos in one- and two-dimensional

coupled map lattices, Physica D 37, 60-82.

- Kaminski, M., Ding, M., Truccolo, W.A., and Bressler, S.T. (2001), Evaluating

causal relations in neural systems: Granger causality, directed transfer

function and statistical assessment of significance, Biol. Cyb. 85, 145-157.

- Lahaye, P.J., Poline, J.B., Flandin, G., Dodel, S., Garnero, L. (2003),

Functional connectivity: studying nonlinear delayed integrations between

BOLD signals, Neuroimage 20, 962-974.

- Levy, P. (1956), Sur une classe de courbes de l’espace de Hilbert et sur une

equation integrale non lineaire, Ann. Sci. Ecole Norm. Sup. 73, 121-156.

- McIntosh, A.R., and Gonzalez-Lima, F. (1994), Structural equation modeling

and its application to network analysis in functional brain imaging, Human

Brain Mapping 2, 2-22.

- Moddemeijer, R. (1999), A statistic to estimate the variance of the histogram-

based mutual information estimator based on dependent pairs of observations,

Signal Proc. 75, 51-63.

- Moon, Y., Rajagopalan, B., and Lall, U. (1995), Estimation of mutual

information using kernel density estimators, Phys. Rev. E 52, 2318-2321.

- Ozaki, T. (1992), Identification of nonlinearities and non-Gaussianities in time

series, in New Directions in Time Series Analysis, Part I, D.R. Brillinger, P.

Caines, J. Geweke, E. Parzen, M. Rosenblatt and M.S. Taqqu, Eds., IMA

36

Volumes in Mathematics and Its Applications, Volume 45, Springer, 227-264.

- Ozaki, T., and Iino, M. (2000), An innovation approach to non-Gaussian time

series analysis, J. Applied Probability Trust 38A, 78-92.

- Ozaki, T., Jimenez, J.C., Peng, H., and Haggan-Ozaki, V. (2004), The

innovation approach to the identification of nonlinear causal models in time

series analysis, in Time Series Analysis and Applications to Geophysical

Systems, D.R. Brillinger, E.A. Robinson and F.P. Schoenberg, Eds., IMA

Volumes in Mathematics and Its Applications, Volume 139, Springer, 195-226.

- Prichard, D., and Theiler, J. (1995), Generalized redundancies for time series

analysis, Physica D 84, 476-493.

- Protter, P. (1990), Stochastic Integration and Differential Equations, Springer,

Berlin.

- Priestley, M.B. (1988), Nonlinear and Non-stationary Time Series Analysis,

Academic Press, San Diego.

- Riera, J., Watanabe, J., Kazuki, I., Naoki, M., Aubert, E., Ozaki, T., and

Kawashima, R. (2004a), A state-space model of the hemodynamic approach:

nonlinear filtering of BOLD signals, Neuroimage, 21, 547-567.

- Riera-Diaz, J.J.; Bosch-Bayard, J.; Yamashita, O.; Kawashima, R.; Sadato, N.;

Okada, T.; Ozaki, T. (2004b), fMRI activation maps based on the NN-ARx

model.Neuroimage, 23(2), (2004), 680-697.

- Riera, J., Aubert, E., Iwata, K., Kawashima, R., Wan, X., and Ozaki, T. (2005),

Fusing EEG and fMRI based on a bottom-up model: Inferring activation and

effective connectivity in neural masses, Phil. Trans. R. Soc. Lond. B., 360,

1457, 1025-1041, 2005.

- Shibata, R. (1980), Asymptotically efficient selection of order of the model for

estimating parameters of a linear process, Annals of Statistics 8, 147-164.

- Shannon, C.E. (1948), A mathematical theory of communication, Bell Syst.

Tech. J. 27, 379-423 and 623-656.

- Valdes, P.A., Jimenez, J.C., Riera, J., Biscay, R. and Ozaki, T. (1999),

Nonlinear EEG Analysis based on a neural Mass Model, Biol. Cyb., 81, 415-

424.

37

- Wiener, N. (1956), Theory of Prediction, in Modern Mathematics for Engineers,

E.F. Beckenbach, Ed., McGraw-Hill, New York, chapter 8.

- Yamashita, O., Galka, A., Ozaki, T., Biscay, R., and Valdes-Sosa, P. (2004),

Recursive penalized least squares solution for dynamical inverse problems of

EEG generation, Human Brain Mapping 21, 221-235.

- Yamashita, O., Sadato, N., Okada, T., and Ozaki, T. (2005), Evaluating

directed connectivity of BOLD signals applying the multivariate autoregressive

model and relative power contribution NeuroImage 25, 478– 490.