Blind Signal Separation & Multivariate Non-Linear Dependency Measures - Peter Addo. December, 15 2014

Blind Signal Separation &

Multivariate Non-Linear

Dependency Measures

SYstemic Risk TOmography:

Signals, Measurements, Transmission Channels, and Policy Interventions

Peter Martey Addo (Speaker) (Centre National De La Recherche Scientifique (CNRS), Universite Paris1 Pantheon-Sorbonne ) Hayette Gatfaoui (NEOMA Business School) Philippe de Peretti (Universite Paris1 Pantheon-Sorbonne) CSRA research meeting – December, 15 2014

Outline: a package of (2) methodologies ©

1 Part I: Causal dependencies in multivariate time seriesTime series graphs & information theoretic measure

2 Part II: Tracking dependency in large multivariate financial systems.Coupling & decoupling information

Peter Martey Addo (joint with Hayette Gatfaoui & Philippe de Peretti) Centre National De La Recherche Scientifique Universite Paris1 Pantheon-Sorbonne NEOMA Business School The Consortium for Systemic Risk Analytics (CSRA) Massachusetts Institute of Technology (MIT ) Cambridge, MassachusettsInformation theory, Econometrics and Systemic risk December 15, 2014 2 / 20

http://pkaddo2000.weebly.com/

Part I: Causal dependencies in multivariate time series Time series graphs & information theoretic measure

Part I: Multivariate Dependency Measures

“The kiss of information theory that captures systemic risk tomography” (jointwith Philippe de Peretti)

Detection and quantification of causal dependencies in multivariate timeseries

How do we create a formal measure of systemic risk that adequately capturescomplex linkages in the financial system? ©

Granger causality & transfer entropy ??



Introduction

Drawbacks of Granger causality & transfer entropy §

Transfer entropy is the information-theoretic analogue of Granger causality

It reduces to Granger causality for vector auto-regressive processes

It is advantageous for the analysis of non-linear signals where the modelassumption of Granger causality might not hold.

The transfer entropy is not uniquely determined by the interaction of the twocomponents alone and depends on misleading effects such as autodependencyand interaction with other process.

It requires arbitrary truncation during estimation, it usually requires moresamples for accurate estimation

Can lead to false interpretation since it is not lag-specific.



Introduction

Our Offer ©

A novel information theoretic model-free approach to understanding systemicrisk

To detect and quantify causal dependencies from multivariate time series.

It gives similar scores to equally noisy dependencies.

It is uniquely determined by the interaction of the two components alone andin a way autonomous of their interaction with the remaining process.

Excludes the misleading influence of autodependency within a process

Lag-specific. Enhance better interpretation.

Only requires that the multivariate time series be stationary.



Overview of approach

Methodology

Let X be a multivariate time series with a set of subprocesses V at each timet ∈ Z and directional links be defined in E . Then

G = (V × Z,E )

is the time series graph of X, where the set of nodes in the graph are made up ofV .

Like graphical models (Lauritzen 1996), TSG’s are based on the concept ofconditional independence.

Note that the time-dependence in the time series is used to define directionallinks in the graph.




Definition (Lag-specific directed link)

We say that the nodes At−τ ∈ G and Bt ∈ G are connected by a lag-specificdirected link “At −→τ Bt” pointing forward in time if and only if τ > 0 and

IA link−→B

(τ) ≡ I(At−τ ;Bt

∣∣ X−\{At−τ})> 0. (1)

Thus, At−τ and Bt are connected if they are not independent conditionally on thepast of the whole process excluding {At−τ} (denoted by the symbol \) which

implies a lag-specific causality with respect to X. I(·; ·| ·

)denotes conditional

mutual information.

If A 6= B then “At −→τ Bt” represents a coupling at lag τ .

An autodependency at lag τ corresponds to A = B.




Definition (Undirected contemporaneous link)

The nodes At ∈ G and Bt ∈ G are connected by an undirected contemporaneouslink “A− B” if and only if

IA

link−B≡ I(At ;Bt

∣∣ X−t+1\{At ,Bt})> 0. (2)

Definition (Notion of parents and neighbors of subprocesses.)

Given the nodes At ∈ G and Bt ∈ G, the parents PBt and neighbors NBt of nodeBt are defined as

PBt ≡ {At−τ : A ∈ X, τ > 0,At−τ −→ Bt} (3)

NBt ≡ {At : A ∈ X,At − Bt} (4)

Definition (Causal Markov Condition)

Any node Bt ∈ G in the time series graph is conditionally independent of X−t \PBt

given its parents PBt .




Definition (Momentary Information Transfer (MIT) Links)

For two subprocesses A, B of a stationary multivariate discrete time process Xwith parents PAt and PBt in the associated time series graph and τ > 0, thegeneral information theoretic measure between nodes At−τ and Bt is given by

IAMIT−→B

(τ) ≡ I(At−τ ;Bt

∣∣ PBt\{At−τ},PAt−τ

)> 0

= H(Bt∣∣ PBt\{At−τ},PAt−τ

)− H

(Bt∣∣ PBt

)and contemporaneous MIT defined by

IA

MIT− B≡ I(At ;Bt

∣∣ PBt ,PAt ,NAt\{Bt},NBt\{At},PNAt \{Bt},PNBt \{At}

). (5)

where H(X ) is Shannon’s entropy and H(X|Y) denotes conditional Shannon’sentropy.

The parents of all subprocesses in X together with the contemporaneous linksforms the time series graph.




Definition (Coupling Strength)

The partial correlation MIT measure, denoted ρMIT , associated with equation (5),for the strength of coupling mechanism between At−τ and Bt is given by

ρAMIT−→B

(τ) ≡ ρ(At−τ ;Bt

∣∣ PBt\{At−τ},PAt−τ

). (6)

The measure ρMIT quantifies how much the variability in A at the exact lag τdirectly influences B, irrespective of the past of At−τ and Bt .

ρMIT is the cross-correlation of the residual after At−τ and Bt have beenregressed on both the parents of At−τ and Bt .

The contemporaneous MIT in the linear case is equivalent to the partialcorrelation of the errors after regressing each process on its parents.

Unlike classical statistics, interactions in the framework of information theoryare viewed as transfers of information and thus this approach is model-free.




Estimation

Summary

In the first Step, PC algorithm (Spirtes et al [SSCR 1991]) is used toestimate the parents of each process, i.e., as a variable selection method.

Unlike graphical models, only undirected links are inferred and the second stepof PC algorithm is omitted.

This first step determines the existence or absence of a link, which also provideuseful information on the causality between lagged components of themultivariate process.

In the second step, MIT is used and all possible links are tested again.

In this step, the problem of serial dependencies is drastically reduced usingMIT (Runge et al[Physical Review E, 2012]).




Simulation Exercise

Consider a simulated 1000 points of a stationary multivariate autoregressiveprocess made up of four subprocesses {Xt ,Yt ,Zt ,Wt}

′defined by

Xt = aXt−1 + cZt−4 + εx (7)

Yt = kXt−1 + hYt−1 + εy (8)

Zt = dYt−2 + bZt−1 + fWt−1 + εz (9)

Wt = eYt−3 + gWt−1 + εw (10)

and the innovation covariance matrix given by Σε =

1 0 d 00 1 0 dd 0 1 00 d 0 1

, where

a = 0.6, b = 0.4, c = 0.3, d = −0.3, e = −0.6, f = 0.2, g = 0.4, k = 0.3, andh = 0.6.

Notice that the lagged causal chain for this process is X −→1 Y −→2 Z withfeedback Z −→4 X , and Y −→3 W −→1 Z , plus contemporaneous links X − Zand Y −W .




X

Y

Z

W

12

3

41

0.0 0.2 0.4 0.6 0.8 1.0

Auto-MIT1.0 0.8 0.6 0.4 0.2 0.0 0.2 0.4 0.6 0.8 1.0

Cross-MIT

0.60.40.20.00.20.40.6

X

→ X → Y → Z → W

0.60.40.20.00.20.40.6

Y

0.60.40.20.00.20.40.6

Z

0 1 2 3 4 5 60.60.40.20.00.20.40.6

W

0 1 2 3 4 5 6 0 1 2 3 4 5 6 0 1 2 3 4 5 6

ρMIT

lag τ [units]

Figure : MIT plot for the simulated process. The plot of significant lags for simulatedprocess associated with the MIT plot.

The results indicates a lagged causal chain as X −→1 Y −→2 Z with feedbackZ −→4 X , and Y −→3 W −→1 Z , plus contemporaneous links X − Z andY −W .




Research-in-progress

Develop nonlinear equivalent measure of the coupling strength.

Empirical application: Causal dependencies in financial institutions

Further Application: Sovereign CDS, Credit Risk etc



Part II: Tracking dependency in large multivariate financial systems. Coupling & decoupling information

Part II: Blind Signal Separation

Tracking dependency in large multivariate financial systems.

Track dependency in large multivariate financial systems.

Study the time-varying information coupling and decoupling

Deduce measures of excess dependency, frailty of the market or multivariatefinancial risk

Deduce Early Warning Indicators.

Build dependency measures for multivariate systems that exhibit :

Rapid changing dynamics, i.e. exhibit a high degree of non-linearity

Non-Gaussianity

Non-stationarity.




Independent Component Analysis (ICA) ©

Assume that the dynamics of the multivariate signal is driven by latentindependent non-gaussian signals

Let {Xt}Tt=1 be a multivariate process with d ∈ Z+ components such that

Xt = ASt (11)

where Xt is the observed process, St are the unobserved independent signals.St signals, as well as the unmixing matrix W :

St =WXt

Thus W = A−1 will contain all relevant information about the dependencystructure of the system, revealed by off-diagonal elements.




The Idea is to build information coupling measures entirely based on W.

Here we have full information decoupling if W is the identity matrix, andinformation coupling between some components if some elements in rows arenon-zero.

Then given this measures, use rolling windows to study how the couplinginformation evolves over time. Uses it to develep new risk measures or earlywarnings.

The mixing process A might change with time - thus a dynamic mixingprocess can arise due to regime changes or structural changes.

Note that mixing of the latent signals does not need to be linear.




Research continues

“If you’re not prepared to be wrong, you’ll never come up with anything original.”(Sir Ken Robinson at TED 2006)


http://www.ted.com/talks/ken_robinson_says_schools_kill_creativity



EU’s Seventh Framework Programme (SYRTO Project)

This project has received funding from the European Union’s Seventh FrameworkProgramme (FP7-SSH/2007-2013) for research, technological development anddemonstration under grant agreement no320270 (SYRTO)


http://syrtoproject.eu/



Thank you for your attention ©



This project has received funding from the European Union’s

Seventh Framework Programme for research, technological

development and demonstration under grant agreement n° 320270

www.syrtoproject.eu

This document reflects only the author’s views.

The European Union is not liable for any use that may be made of the information contained therein.

Economy & Finance

Blind Signal Separation & Multivariate Non-Linear Dependency Measures - Peter Addo. December, 15 2014