Model-based and model-free connectivity methods for electrical neuroimaging

Directed dynamical connectivity in electricalneuroimaging: which tools should I use?

A very partial and personal overview, in good faith but still

Daniele Marinazzo

Department of Data Analysis, Faculty of Psychology and Educational Sciences,Ghent University, Belgium

7 @dan marinazzohttp://users.ugent.be/~dmarinaz/

Daniele Marinazzo Directed connectivity in electrical neuroimaging

http://users.ugent.be/~dmarinaz/

At least two distinct ways one can think of causality

Temporal precedence, i.e. causes precede their consequences

Physical influence (control), i.e. changing causes changes theirconsequences


At least two distinct ways one can think of causality

Temporal precedence, i.e. causes precede their consequences

Physical influence (control), i.e. changing causes changes theirconsequences


Two classes of methods

Assume independent measurements at each node

Inference of networks from temporally correlated data (dynam-ical networks)


Using temporal dynamics

We model a dynamical system at each node

Two main approaches:

Dynamic Bayesian networks (Hidden Markov Models)

Model-free and model-based investigation of temporal correla-tion


What to expect from ”causality” measures in neuroscience

Causal measures in neuroscience should reflect effective con-nectivity, i.e. the underlying physiological influences exertedamong neuronal populations in different brain areas. → Dy-namic Causal Models

Different but complementary goal: to reflect directed dynam-ical connectivity without requiring that the resulting networksrecapitulate the underlying physiological processes. → GrangerCausality, Transfer Entropy

The same underlying (physical) network structure can give riseto multiple distinct dynamical connectivity patterns

In practice it is always unfeasible to measure all relevant vari-ables

Bressler and Seth 2010
















Basic idea of Dynamic Causal Models

We have several neural populations ..



.. with interactions among and within them



What we see and what we don’t



Forward model



Bayesian framework



Bayesian framework



Model inference

Prior: what connections are included in the model

Likelihood: Incorporates the generative model and predictionerrors

Model evidence: Quantifies the goodness of a model (i.e.,accuracy minus complexity). Used to draw inference on modelstructure.

Posterior: Probability density function of the parameters giventhe data and model. Used to draw inference on model param-eters.



Inference on model structure

Which model (or family of models) has highest evidence?

Inference on model parameters

Which parameters are statistically significant, and what is theirsize/sign?




Which model (or family of models) has highest evidence?


Which parameters are statistically significant, and what is theirsize/sign?



A necessary step, unless strong prior knowledge about structure

Bayesian model comparison (BMS) compares the (log) modelevidence of different models (i.e., probability of the data givenmodel)

log model evidence is approximated by free energy

The Kullback - Leibler divergence between the real and approx-imate conditional density minus the log-evidence

A Bayesian Expectation Maximization

ok, a model fit








ok, a model fit








ok, a model fit








ok, a model fit



Often a second step in DCM studies

Inference on the parameters of the clear winning model (if thereis one)

If no clear winning model (or if optimal model structure differsbetween groups) then Bayesian model averaging (BMA) isan option

Final parameters are weighted average of individual model pa-rameters and posterior probabilities


Group level inference

Different DCMs are fitted to the data for every subject.

Group inference on the models (or groups of models: in DCMterminology families of models e.g. all models with input toregion A vs. input to region B, or vs. both, three families):Bayesian model selection

Winning model/family is the one with highest exceedance prob-ability

Group inference on model parameter: Either on the winningmodel or Bayesian model averaging (BMA) across models (withina winning family or all models when BMS reveal no clear win-ner)

(BMA) Parameter(s) of interest are harvested for every subjectand subjected to frequentist inference (e.g. t-test)


DCM for ERPs/ERFs

Bottom-up: connection from low to high hierarchical areastop-down: connection from high to low hierarchical areas (Felle-man 1991)

Lateral: same level in hierarchical organization (e.g. interhemi-spheric connection)

Prior on connection: forward → backward → lateral

Layers within regions interact via intrinsic connections


DCM inference: summary



Influences in multivariate datasets

We must condition the measure to the effect of other variables

The most straightforward solution is the conditioned approach,starting from Geweke et al 1984


Influences in multivariate datasets

We must condition the measure to the effect of other variables

The most straightforward solution is the conditioned approach,starting from Geweke et al 1984


Beyond conditioning: joint information


Transfer entropy and Markov property

Absence of causality: generalized Markov property

p(x |X ,Y ) = p(x |X )

Transfer Entropy

Transfer entropy (Schreiber 2000) quantifies the violation of thegeneralized Markov property

T (Y → X ) =

∫p(x |X ,Y ) log

p(x |X ,Y )

p(x |X )dx dX dY

T measures the information flowing from one series to the other.


Transfer entropy and regression

Risk functional

The minimizer of the risk functional

R [f ] =

∫dX dx (x − f (X ))2 p(X , x)

represents the best estimate of x given X , and corresponds to theregression function

f ∗(X ) =

∫dxp(x |X ) x



Markov property for uncorrelated variables

The best estimate of x , given X and Y is now:

g∗(X ,Y ) =

∫dxp(x |X ,Y ) x

p(x |X ,Y ) = p(x |X )⇒ f ∗(X ) = g∗(X ,Y )

and the knowledge of Y does not improve the prediction of x



Transfer entropy (entropy rate)

SX = −∫

dx dX p(x ,X ) log[p(x |X )]

SXY = −∫

dx dX dY p(x ,X ,Y ) log[p(x |X ,Y )]

Regression

EX =

∫dx dX p(x ,X ) (x −

∫dx ′ p(x ′|X ) x ′)2

EX ,Y =

∫dx dX dY p(x ,X ,Y ) (x −

∫dx ′ p(x ′|X ,Y ) x ′)2


Granger causality and Transfer entropy

GC and TE are equivalent for Gaussian variables and otherquasi-Gaussian distributions(Barnett et al 2009, Hlavackova-Schindler 2011, Barnett andBossomaier 2012)

In this case they both measure information transfer.

Unified approach (model based and model free)

Mathematically more treatable

Allows grouping variables according to their predictive content(Faes et al. 2014)


Granger causality and Transfer entropy

GC and TE are equivalent for Gaussian variables and otherquasi-Gaussian distributions(Barnett et al 2009, Hlavackova-Schindler 2011, Barnett andBossomaier 2012)

In this case they both measure information transfer.

Unified approach (model based and model free)

Mathematically more treatable

Allows grouping variables according to their predictive content(Faes et al. 2014)


Joint information

Let’s go for an operative and practical definition

Relation (B and C) → A

synergy: (B and C) contributes to A with more informationthan the sum of its variables

redundancy: (B and C) contributes to A with less informationthan the sum of its variables

Stramaglia et al. 2012, 2014, 2016


Joint information







Joint information







Generalization of GC for sets of driving variables

Conditioned Granger Causality in a multivariate system

δX(B → α) = logε (xα|X \ B)

ε (xα|X)

Unnormalized version

δuX(B → α) = ε (xα|X \ B)− ε (xα|X)

An interesting property

If {Xβ}β∈B are statistically independent and their contributions in

the model for xα are additive, then δuX(B → α) =∑β∈B

δuX(β → α).

This property does not hold for the standard definition of GC, neitherfor entropy-rooted quantities, because logarithm.





ε (xα|X)






δuX(β → α).






ε (xα|X)






δuX(β → α).



Question from the audience:

What does it ever mean to have an unnormalized measure ofGranger causality?

Don’t you lose any link with information?


Question from the audience:

What does it ever mean to have an unnormalized measure ofGranger causality?

Don’t you lose any link with information?


Define synergy and redundancy in this framework

Synergy

δuX(B → α) >∑β∈B δ

uX\B,β(β → α)

Redundancy

δuX(B → α) <∑β∈B δ

uX\B,β(β → α)

Stramaglia et al. IEEE TransBiomed. Eng. 2016

Pairwise syn/red index

ψα(i , j) = δuX\j(i → α)− δuX(i → α)

= δuX({i , j} → α)− δuX(i → α)− δuX(j → α)



Synergy


uX\B,β(β → α)

Redundancy


uX\B,β(β → α)







Synergy


uX\B,β(β → α)

Redundancy


uX\B,β(β → α)






Do it yourself!

Statistical Parametric Mapping - DCM http://www.fil.ion.

ucl.ac.uk/spm/

MVGC (State-Space robust implementation) http://users.

sussex.ac.uk/~lionelb/MVGC/

BSmart (Time-varying, Brain-oriented) http://www.brain-smart.org/

MuTE (Multivariate Transfer Entropy, GC in the covariancecase) http://mutetoolbox.guru/

emVAR (Frequency Domain) http://www.lucafaes.net/emvar.html

ITS (Information Dynamics) http://www.lucafaes.net/its.html


http://www.fil.ion.ucl.ac.uk/spm/

http://www.fil.ion.ucl.ac.uk/spm/

http://users.sussex.ac.uk/~lionelb/MVGC/

http://users.sussex.ac.uk/~lionelb/MVGC/

http://www.brain-smart.org/

http://www.brain-smart.org/

http://mutetoolbox.guru/

http://www.lucafaes.net/emvar.html

http://www.lucafaes.net/emvar.html

http://www.lucafaes.net/its.html

http://www.lucafaes.net/its.html

Thanks

Hannes Almgren, Ale Montalto and Frederik van de Steen (UGent)

Sebastiano Stramaglia (Bari)

Pedro Valdes Sosa (CNeuro and UESTC)

Laura Astolfi and Thomas Koenig


References

David et al., 2006: Dynamical causal modelling of evoked reponses in EEG and MEG (NI)

Stephan et al., 2010: Ten simple rules for dynamic causal modeling (NI)

Penny et al., 2004: Comparing Dynamic causal models (NI)

Litvak et al., 2008: EEG and MEG Data Analysis in SPM8 (CIN)

Bressler and Seth, 2010: Wiener-Granger causality, a well-established methodology (NI)

Montalto et al., 2014: MuTE: A MATLAB Toolbox to Compare Established and Novel Estimators of theMultivariate Transfer Entropy (PLOS One)

Bastos and Schoffelen, 2016: A Tutorial Review of Functional Connectivity Analysis Methods and TheirInterpretational Pitfalls (Front N Sys)

Stramaglia et el. 2106: Synergetic and Redundant Information Flow Detected by Unnormalized GrangerCausality (IEEE TBME)
