12
Quantitative and qualitative analysis of asynchronous neural activity Ekkehard Ullner, Antonio Politi Institute for Pure and Applied Mathematics and Department of Physics (SUPA), Old Aberdeen, Aberdeen AB24 3UE, United Kingdom Alessandro Torcini Laboratoire de Physique Th´ eorique et Mod´ elisation, Universit´ e de Cergy-Pontoise, CNRS, UMR 8089,95302 Cergy-Pontoise cedex, France and Istituto dei Sistemi Complessi, CNR - Consiglio Nazionale delle Ricerche, via Madonna del Piano 10, I-50019 Sesto Fiorentino, Italy The activity of a sparse network of leaky integrate-and-fire neurons is carefully revisited with reference to a regime of a bona-fide asynchronous dynamics. The study is preceded by a finite- size scaling analysis, carried out to identify a setup where collective synchronization is negligible. The comparison between quenched and annealed networks reveals the emergence of substantial differences when the coupling strength is increased, via a scenario somehow reminiscent of a phase transition. For sufficiently strong synaptic coupling, quenched networks exhibit a highly bursting neural activity, well reproduced by a self-consistent approach, based on the assumption that the input synaptic current is the superposition of independent renewal processes. The distribution of interspike intervals turns out to be relatively long-tailed; a crucial feature required for the self- sustainment of the bursting activity in a regime where neurons operate on average (much) below threshold. A semi-quantitative analogy with Ornstein-Uhlenbeck processes helps validating this interpretation. Finally, an alternative explanation in terms of Poisson processes is offered under the additional assumption of mutual correlations among excitatory and inhibitory spikes. I. INTRODUCTION The characterization of the spiking activity of neuronal networks is a long standing problem even with reference to the asynchronous regime: simple from a dynamical point of view, but extremely relevant for understanding cortex dynamics [1, 2]. A moment’s reflection indeed suggests that this is not a trivial task whenever the self- generated neuron-input current is not constant: to what extent can the fluctuations be treated as a stochastic pro- cess? One of the most popular models used to study neural dynamics consists of two coupled populations of excita- tory and inhibitory leaky integrate-and-fire (LIF) neu- rons accompanied by refractoriness and delay [3, 4]. In- cidentally, this model was also proposed for the char- acterization of asynchronous dynamics in balanced net- works [5]. In a ground breaking paper, Brunel [3] pro- posed to treat the input current as a δ-correlated Gaus- sian process [6], thereby deriving and solving analytically a self-consistent Fokker-Planck equation. Although this approach turns out to be quantitatively accurate for rel- atively small coupling strengths, the same is no longer true for stronger coupling (see [7]), when large deviations from the theoretical predictions are observed. These de- viations may in principle originate from various sources: (i) the spontaneous onset of irregular collective dynamics, which has been found even for relatively small network connectivities [7, 8]; (ii) the non Poissonian nature of the spiking activity; (iii) large amplitude of the single spikes and the consequential possible failure of a perturbative, linear approach; (iv) the presence of non-negligible finite- time correlations. Several alternative approaches have been indeed pro- posed. For instance an exact treatment of shot-noise, for spike amplitudes not vanishingly small, which leads to a mixed Fokker-Planck/master-equation formalism (see [9, 10]). Unfortunately, we are not aware of any way to make the approach self-consistent, by inferring the input properties on the basis of the observed output. Anyhow, since this approach assumes a Poissonian dis- tribution of the inter-spike-intervals – a property largely unsatisfied for large synaptic coupling – one should any- way look for different approximation schemes. A different strategy was proposed by Dummer et al. [11], based on the self-consistent derivation of the power spectrum of the spiking activity. The advantage of this method is that no assumption is made on the spec- tral shape of the synaptic current. While the original im- plementation proved unstable already for relatively small coupling strengths, the variant recently proposed in [12] leads to seemingly accurate reproduction of the network dynamics. We shall treat it as a reference for some of our considerations. In this paper we revisit the problem, starting from the accuracy of numerical simulations and the presence of finite-size corrections. In Refs. [7, 8], it was indeed shown that the firing activity of a network of 10,000 neurons with a in-degree K =1, 000 is strongly affected by the presence of collective irregular dynamics. Our first goal has been therefore that of finding the minimal network- size such that collective effects are negligible. The first result is that quenched networks (charac- terized by a fixed adjacency matrix) exhibit a substan- tially different behaviour from annealed ones (where post-synaptic neighbours are randomly selected when- . CC-BY-NC-ND 4.0 International license available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint this version posted December 20, 2019. ; https://doi.org/10.1101/2019.12.19.882456 doi: bioRxiv preprint

Quantitative and qualitative analysis of asynchronous neural activity · 2019. 12. 19. · Quantitative and qualitative analysis of asynchronous neural activity ... via a scenario

  • Upload
    others

  • View
    4

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Quantitative and qualitative analysis of asynchronous neural activity · 2019. 12. 19. · Quantitative and qualitative analysis of asynchronous neural activity ... via a scenario

Quantitative and qualitative analysis of asynchronous neural activity

Ekkehard Ullner, Antonio PolitiInstitute for Pure and Applied Mathematics and Department of Physics (SUPA),

Old Aberdeen, Aberdeen AB24 3UE, United Kingdom

Alessandro TorciniLaboratoire de Physique Theorique et Modelisation, Universite de Cergy-Pontoise,

CNRS, UMR 8089,95302 Cergy-Pontoise cedex, France andIstituto dei Sistemi Complessi, CNR - Consiglio Nazionale delle Ricerche,

via Madonna del Piano 10, I-50019 Sesto Fiorentino, Italy

The activity of a sparse network of leaky integrate-and-fire neurons is carefully revisited withreference to a regime of a bona-fide asynchronous dynamics. The study is preceded by a finite-size scaling analysis, carried out to identify a setup where collective synchronization is negligible.The comparison between quenched and annealed networks reveals the emergence of substantialdifferences when the coupling strength is increased, via a scenario somehow reminiscent of a phasetransition. For sufficiently strong synaptic coupling, quenched networks exhibit a highly burstingneural activity, well reproduced by a self-consistent approach, based on the assumption that theinput synaptic current is the superposition of independent renewal processes. The distribution ofinterspike intervals turns out to be relatively long-tailed; a crucial feature required for the self-sustainment of the bursting activity in a regime where neurons operate on average (much) belowthreshold. A semi-quantitative analogy with Ornstein-Uhlenbeck processes helps validating thisinterpretation. Finally, an alternative explanation in terms of Poisson processes is offered under theadditional assumption of mutual correlations among excitatory and inhibitory spikes.

I. INTRODUCTION

The characterization of the spiking activity of neuronalnetworks is a long standing problem even with referenceto the asynchronous regime: simple from a dynamicalpoint of view, but extremely relevant for understandingcortex dynamics [1, 2]. A moment’s reflection indeedsuggests that this is not a trivial task whenever the self-generated neuron-input current is not constant: to whatextent can the fluctuations be treated as a stochastic pro-cess?

One of the most popular models used to study neuraldynamics consists of two coupled populations of excita-tory and inhibitory leaky integrate-and-fire (LIF) neu-rons accompanied by refractoriness and delay [3, 4]. In-cidentally, this model was also proposed for the char-acterization of asynchronous dynamics in balanced net-works [5]. In a ground breaking paper, Brunel [3] pro-posed to treat the input current as a δ-correlated Gaus-sian process [6], thereby deriving and solving analyticallya self-consistent Fokker-Planck equation. Although thisapproach turns out to be quantitatively accurate for rel-atively small coupling strengths, the same is no longertrue for stronger coupling (see [7]), when large deviationsfrom the theoretical predictions are observed. These de-viations may in principle originate from various sources:(i) the spontaneous onset of irregular collective dynamics,which has been found even for relatively small networkconnectivities [7, 8]; (ii) the non Poissonian nature of thespiking activity; (iii) large amplitude of the single spikesand the consequential possible failure of a perturbative,linear approach; (iv) the presence of non-negligible finite-time correlations.

Several alternative approaches have been indeed pro-posed. For instance an exact treatment of shot-noise,for spike amplitudes not vanishingly small, which leadsto a mixed Fokker-Planck/master-equation formalism(see [9, 10]). Unfortunately, we are not aware of anyway to make the approach self-consistent, by inferringthe input properties on the basis of the observed output.Anyhow, since this approach assumes a Poissonian dis-tribution of the inter-spike-intervals – a property largelyunsatisfied for large synaptic coupling – one should any-way look for different approximation schemes.

A different strategy was proposed by Dummer etal. [11], based on the self-consistent derivation of thepower spectrum of the spiking activity. The advantage ofthis method is that no assumption is made on the spec-tral shape of the synaptic current. While the original im-plementation proved unstable already for relatively smallcoupling strengths, the variant recently proposed in [12]leads to seemingly accurate reproduction of the networkdynamics. We shall treat it as a reference for some of ourconsiderations.

In this paper we revisit the problem, starting from theaccuracy of numerical simulations and the presence offinite-size corrections. In Refs. [7, 8], it was indeed shownthat the firing activity of a network of 10,000 neuronswith a in-degree K = 1, 000 is strongly affected by thepresence of collective irregular dynamics. Our first goalhas been therefore that of finding the minimal network-size such that collective effects are negligible.

The first result is that quenched networks (charac-terized by a fixed adjacency matrix) exhibit a substan-tially different behaviour from annealed ones (wherepost-synaptic neighbours are randomly selected when-

.CC-BY-NC-ND 4.0 International licenseavailable under a(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made

The copyright holder for this preprintthis version posted December 20, 2019. ; https://doi.org/10.1101/2019.12.19.882456doi: bioRxiv preprint

Page 2: Quantitative and qualitative analysis of asynchronous neural activity · 2019. 12. 19. · Quantitative and qualitative analysis of asynchronous neural activity ... via a scenario

2

ever a spike is emitted). Quite interestingly, the differ-ence emerges almost abruptly above a “critical” synapticcoupling strength (namely, J ≈ 0.25). The existence oftwo seemingly different phases was already claimed byOstojic [4], but challenged in Ref. [13]. Here, we do notinvestigate the behaviuor in the vicinity of the hypothet-ical phase transition, but rather focus on the characteri-zation of the large-coupling regime, since we believe thatthe accurate characterization of a given phase has higherpriority.

One of the main results of this paper is that the synap-tic current can be accurately represented as the superpo-sition of independent identical renewal processes, eachcharacterized by a suitable interspike-interval (ISI) dis-tribution. We also show that the correlations due tothe long-tailed ISI distribution can be equivalently rep-resented as long-term memory in the symbolic represen-tation of inhibitory vs excitatory spikes.

More precisely, in Sec II, we introduce the model anddefine the indicators later used to characterize and dis-cuss the various dynamical properties. In the followingSec III we illustrate the firing activity of the quenchednetwork, computing several indicators for different cou-pling strengths. A relatively quick discussion is alsodevoted to the annealed set-up to show the differenceswith respect to the quenched case. In Sec. IV we firstintroduce the two self-consistent approaches herein im-plemented to characterize the neural activity. The for-mer one, based on the distribution of ISIs, provides arather accurate description. The latter, already proposedin [11, 12], reveals an unexpectedly stable fixed point,which, however, is further away from the results of ac-curate simulations. In Sec. V, we turn our attention tothe bursting activity observed for large coupling in theattempt of explaining how neurons operating on averagebelow threshold are able to exhibit a strong firing activ-ity. Finally, in Sec. VI we summarize the main resultsand focus on the still open problems.

II. MODEL AND METHODS

A. Network Model

Due to its relevance in the context of asynchronous dy-namics in balanced networks [3, 4, 14, 15] we considerthe following sparse spiking network of LIF neurons. Thenetwork is composed of bN excitatory and (1 − b)N in-hibitory cells; the membrane potential Vi of the i-th neu-ron evolves according to the equation

τ Vi = R(I0 + Ii)− Vi , (1)

where τ = 20 ms is the membrane time constant,RI0 = 24 mV is an external DC supra-threshold “cur-rent”, while RIi is the synaptic current arising from therecurrent coupling

RIi(t) = τJ∑n

Gij(n)δ(t− t(j)n − τd) , (2)

where J is the coupling strength and the sum runs over all

the spikes emitted at time t(j)n < t from the pre-synaptic

neurons j(n) connected to neuron i. Gij is the adjacencymatrix and its elements assume the following values:Gij = 1 (−g), if the pre-synaptic neuron j is excitatory(inhibitory), otherwise Gij = 0. If Vj reaches the thresh-

old Vth = 20 mV at time t(j)n , two events are triggered:

(i) the membrane potential Vj is reset to Vr = 10 mVand held fixed for a refractory period τr = 0.5 ms; (ii)a spike is emitted and received τd = 0.55 ms later bythe post-synaptic cells connected to neuron j accordingto Gij . Except for the system size N , all parameters areset as in Ref. [4]: b = 0.8, K = 1000, and g = 5, sothat each neuron receives input from bK ((1 − b)K) ex-citatory (inhibitory) pre-synaptic neurons. Besides thisquenched setup, we have considered annealed networks,where the post-synaptic neighbours are randomly chosenat each spike emission. As a matter of fact, in the former(latter) case, the in-degree (out-degree) is equal to K foreach neuron, while the out-degree (in-degree) is binomi-ally distributed with average K and standard deviation√K, in the thermodynamic limit. Our choice was dic-

tated by efficiency of the numerical simulations; howeverwe have verified that no substantial changes are observedif, instead of fixing the number of links equal to K, theprobability of each link is set equal c = K/N , as in trulyErdos-Renyi networks.

B. Methods

A detailed description of network dynamics requireslooking both at the microscopic and the macroscopiclevel.

1. Microscopic Indicators

The dynamics of a spiking neuron is usually charac-terized in terms of the probability distribution function(PDF) Q(T ) of the ISIs T and of the associated moments:namely, the mean ISI T and the standard deviation σTof T . Usually, the regularity/irregularity of the dynamicsis quantified by the so-called coefficient of variation,

Cv =σT

T,

equal to zero for a periodic dynamics and to 1, for Pois-sonian spike trains. It should be noticed that Cv can belarger than 1 for the so-called bursting dynamics, whenthe neuron alternates periods of silence and high activity.The firing rate of a neuron is simply given by ν = 1/T .In order to characterize the network activity we estimatethe mean coefficient of variation 〈Cv〉 and the mean firingrate 〈ν〉, where 〈·〉 represents an ensemble average overall neurons.

An important observable is the power spectrum S(f) =〈|u(f)|2〉, where u(f) with f = m/(Mδt) is the Fourier

.CC-BY-NC-ND 4.0 International licenseavailable under a(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made

The copyright holder for this preprintthis version posted December 20, 2019. ; https://doi.org/10.1101/2019.12.19.882456doi: bioRxiv preprint

Page 3: Quantitative and qualitative analysis of asynchronous neural activity · 2019. 12. 19. · Quantitative and qualitative analysis of asynchronous neural activity ... via a scenario

3

transform of the neural activity u(t), determined by com-puting the number of spikes emitted in M consecutivetime intervals of duration δt. Further observables we fo-cused on are the phase correlations among different fre-quencies f and h, which can be quantified by the nor-malized indicator,

D(f) =

√∑h6=f 〈|u(h) u∗(f)|2〉

W S(f)(3)

where W = 〈∑h |u(h)|2〉 is the total power of the spec-

trum. One can check that 0 ≤ D(f) ≤ 1, the lower(upper) bound corresponding to uncorrelated (perfectlycorrelated) channels. The typical values we have used inour simulations are δt = 0.11 ms and M = 215 (M = 212)for the power spectrum (phase correlations) estimation.

2. Macroscopic Indicators

At the mean-field level, the network evolution is cap-tured by the instantaneous PDF P (v, t) of the mem-brane potentials of the neurons. In the limit case ofan infinitely large in-degree, the perfectly asynchronousregime is characterized by a constant firing rate 〈ν〉 [16].This implies that the flux of neurons along the v-axis isindependent of both time and potential value, i.e. thecorresponding PDF P0(v) should be stationary.

Deviations from stationarity reveal the presence of acollective dynamics. In order to measure the level ofcoherence in the network dynamics, a commonly usedorder parameter is [17]

ρ2 ≡ 〈v〉2 − 〈v〉

2

〈v2 − v2〉; (4)

where the overbar denotes a time average. In practice,ρ is the rescaled amplitude of the standard deviation ofthe average 〈v〉. When all neurons behave in exactly thesame way (perfect synchronization), the numerator andthe denominator are equal to one another and ρ = 1.If instead, they are independent as in an asynchronousregime, ρ ≈ 1/

√N due to the central limit theorem.

III. NETWORK DYNAMICS

A. The quenched network

Our simulations have been mostly performed by im-plementing an exact event-driven scheme; see [8] for adescription of the details. Since, however, some simula-tions required implementing a clock-driven Euler schemeand since this latter approach is often used in the liter-ature, we have first compared the two algorithms for anetwork of 105 neurons with a coupling strength J = 0.8.

0 20 40 60 80integration time [s]

30

40

50

60

ν [

Hz]

_

FIG. 1. Running average of the firing rate in a network ofN = 105 neurons with a coupling strength J = 0.8 and overallconnectivity K = 103. The black solid line refers to event-driven simulations; the dashed lines correspond to Euler in-tegration scheme with different time steps (δt = 0.1 ms, 0.04ms, 0.01 ms, 0.001 ms, from bottom to top).

From the results, reported in Fig. 1, we see that the timestep δt used in the implementation of Euler’s algorithmshould not be larger than 10−3 ms in order to get resultsessentially in agreement with the event driven scheme.This is indeed the value employed in our simulations per-formed with the Euler’s scheme. Notice that for δt = 0.1ms, a value often chosen in the literature, the firing rateis substantially underestimated (by approximately 24%).

We then proceed to analysing the dependence of theaverage firing rate 〈ν〉 on the coupling strength J . Theblack, solid curve in Fig. 2(a) has been obtained forN = 104 and exactly the same parameter values as inRef. [4]. As reported therein, 〈ν〉, after an initial drop,increases with the coupling strength J . One of the goalsof this paper is to understand the origin of this growthin a network where inhibition is expected to prevail overexcitation. A theoretical estimate νT of the average fir-ing rate in the asynchronous regime of a sparse networkcan be derived from the stationary solution of a self-consistent Fokker-Planck equation, under the assump-tion of an uncorrelated Poissonian activity of the neu-rons [3, 6]. However, this prediction, based on the dif-fusion approximation [6] and reported as a dotted greencurve in Fig. 2(a), is able to reproduce only the initialpart of the curve 〈ν(J)〉, while it fails to describe thegrowth observable for larger coupling. Furthermore, inRefs. [7, 8] it was found that the corresponding dynami-cal phase is far from asynchronous; this is testified by thebehavior of ρ(J), reported in Fig. 2(b), where we can seethat the order parameter ρ can be as large as 0.5. Con-sidering that the theoretical prediction has been derivedunder the assumption of a strictly asynchronous regime(i.e. ρ = 0), it is therefore crucial to separate out the

.CC-BY-NC-ND 4.0 International licenseavailable under a(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made

The copyright holder for this preprintthis version posted December 20, 2019. ; https://doi.org/10.1101/2019.12.19.882456doi: bioRxiv preprint

Page 4: Quantitative and qualitative analysis of asynchronous neural activity · 2019. 12. 19. · Quantitative and qualitative analysis of asynchronous neural activity ... via a scenario

4

effects of the collective dynamics.

This can be done by increasing the network size, whileleaving the in-degree fixed (namely, K = 1000). Quitesurprisingly, the firing rate obtained for N = 105, (seethe upper blue solid curve in Fig. 2(a)) displays an evenmore pronounced growth than for N = 104, in spite ofa weaker synchronization, as shown in Fig. 2(b). Theanalysis reported in Fig. 2(d), where ρ vs. c = K/Nis reported for three different synaptic coupling values,shows that the collective effects increase as ρ ≈

√c, con-

sistently with the theoretical expectations [3].

Going back to Fig. 2(a), we see that upon further in-creasing N above 105, the firing rate for a given couplingstrength J saturates to a finite value. Altogether, wecan safely conclude that the increase of ν for J & 0.3is a genuine property of a bona fide asynchronous activ-ity and it should be explained as such. Our simulationssuggest that the system-size N = 105 is large enoughto ensure nearly asymptotic results and small enough toallow for affordable simulation times. From now on, allsimulations will refer to this network size, unless statedotherwise. The main questions we want to address areunderstanding: (i) the features of such a high firing-rateregime and (ii) why it deviates so strongly from the diffu-sion approximation [3, 6] even for a not-too-large synap-tic coupling J in a setup where correlations among thedifferent neurons are practically absent.

Before proceeding along these lines, it is useful to pro-vide a more detailed description of the network activity.In Fig. 2(c) the mean coefficient of variation 〈Cv〉 is plot-ted versus J for different networks sizes. There, we seethat Cv steadily increases with J and converges to someasymptotic value upon decreasing c. In practice, the neu-ral activity can be never treated as a Poissonian process,as requested by the diffusion approximation employedin [3]; it is either more regular (for small coupling), orsubstantially more intermittent (as for J > 0.4). There-fore, it should not come as a surprise that a theoreticalapproach, such as that in Ref. [3, 6], based on the as-sumption that 〈Cv〉 = 1, is not accurate.

Cv gives only a rough information about the distribu-tion of the ISIs. It is worth turning our attention to thefull shape of the ISI distribution Q(T ). In Fig. 3, we plotQ(T ) for J = 0.1, J = 0.5 and J = 0.8; in all cases, wesee that for large enough ISIs, the PDF is characterizedby an exponential tail as for a Poissonian process. How-ever, for small ISIs, the PDF is substantially different.For weak coupling, very small ISIs are strongly inhib-ited [18]: this is an obvious consequence of the nearlyconstant input current (mean driven). For stronger cou-plings, the PDF exhibits a quasi power-law decay whichextends up to 10-20 ms (see the insets of Fig. 3(b) and(c)). These features are consistent with the characteriza-tion of the ISI distributions reported in [19] for spikingneurons driven by fluctuating inputs. In particular, thePDF shown in Fig. 3(a) is expected to emerge when theaverage effective input current (including the contribu-tion of the synaptic coupling) lies between the threshold

0 0.2 0.4 0.8J0

10

20

30

40

50

> [H

z]

0 0.2 0.4 0.8J0

0.2

0.4

ρ

0 0.2 0.4 0.8J0

1

2

3

4

<Cv>

0.001 0.01 0.1c

0.1

b)

d)c)

a)

FIG. 2. Collective properties of the network dynamics ver-sus the coupling strength J : (a) average firing rate 〈ν〉; (b)the coherence order parameter ρ; (c) the mean coefficient ofvariation 〈Cv〉. Solid lines and symbols refer to quenchednetworks (namely, black lines correspond to N = 104; blueones to N = 105 and (red) crosses to N = 8 × 105) (blue)dashed lines correspond to simulations of the annealed net-work performed for N = 105 neurons. The (green) dottedline in panel (a) corresponds to the theoretical prediction νTderived in [3, 6]. Finally, panel (d) displays the coherence or-der parameter ρ versus the connectivity c = K/N , reportingonly results for the quenched network for different synapticcoupling: namely, J = 0.1, J = 0.5, and J = 0.8 (solid linesfrom the bottom to the top). The dotted line refers to thetheoretical scaling law

√c.

and the reset value, which is indeed the case.

Furthermore, we have computed the power spectrumS(f) of single spike sequences. For weak coupling (J =0.1), a small peak is visible at f = 17 Hz in Fig. 4(a) [20]it is reminiscent of the periodic activity of the uncoupledneurons. At higher frequencies, the spectrum is practi-cally flat, i.e. it is approximately white. Upon increas-ing the coupling, the spectrum starts exhibiting a low-frequency peak suggesting the presence of “long”-timecorrelations. This feature will be further discussed inSec. V with reference to the emergence of a bursting ac-tivity. For J = 0.8 subsidiary peaks, related to the delay,emerge for f = 1818 Hz and its multiples. The delayis always present but for unexplained reasons pops uponly for large coupling when the white-noise backgroundis even larger.

B. The annealed network

So far we focused on the dynamical properties of a net-work characterized by a quenched distribution of synap-tic connections. However, all theoretical approaches de-veloped to characterize the firing activity do not take

.CC-BY-NC-ND 4.0 International licenseavailable under a(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made

The copyright holder for this preprintthis version posted December 20, 2019. ; https://doi.org/10.1101/2019.12.19.882456doi: bioRxiv preprint

Page 5: Quantitative and qualitative analysis of asynchronous neural activity · 2019. 12. 19. · Quantitative and qualitative analysis of asynchronous neural activity ... via a scenario

5

0 50 100 200 250T [ms]10

-6

10-5

10-4

10-3

10-2

Q(T)(a)

200 400 800 1000T [ms]

10-5

10-4

10-3

10-2

10-1

Q(T)

100

101

103T [ms]

10-4

10-2

Q(T)(b)

500 1000 1500T [ms]

10-6

10-5

10-4

10-3

10-2

10-1

Q(T)

1 10 1000T [ms]

10-6

10-4

10-2

Q(T)(c)

FIG. 3. The PDF Q(T ) of the ISIs T generated by the fullquenched network (black) and as obtained after the first stepof the renewal process recursive procedure (red curve). Panels(a), (b) and (c) refer to J = 0.1, J = 0.5, and J = 0.8, re-spectively. The insets contain the same information in log-logscales to emphasize the initial quasi-power-law decay. The in-set in panel (c) shows (in green) the output of a single neuronby a symbol-correlated Poisson process (see Sec. V C).

into account the actual, invariant structure of the connec-tions. Even more, theoretical approaches do not includedelay at all. Therefore, it is natural to ask to what extentthe quenched nature of the network may be consideredresponsible for the observed asynchronous activity. Thisquestion can be addressed by considering an annealednetwork, where the “neighbours” of each given neuronare randomly assigned each time a spike is emitted. Moreprecisely, we proceed as follows: whenever a neuron fires,we still assume that the quality of the spike (excitatoryvs inhibitory) is determined by the neuron itself, but we

100

101

102

103

f [Hz]

101

102

103

S(f

) [

Hz]

(a)

100

101

102

103

f [Hz]

101

102

103

S(f

) [H

z]

(b)

100

101

102

103

f [Hz]

101

102

103

S(f

) [H

z]

(c)

FIG. 4. Power spectra S(f) of the neural activity. Blackcurves refer to the full quenched network for N = 105; redcurves refer to the first step of the renewal process approxi-mation. Panels (a), (b), and (c) refers to J = 0.1, 0.5, and0.8, respectively.

randomly choose K receiving neurons regardless of theirquality. Moreover, we exclude self-connections, i.e. thesender must differ from the receiver. Finally, we keep allparameters as in the quenched network. This guaran-tees that on average each neuron receives bK excitatoryinputs and (1− b)K inhibitory ones.

The numerical results for the average firing rate are re-ported in Fig. 2(a). There we observe a good agreement

.CC-BY-NC-ND 4.0 International licenseavailable under a(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made

The copyright holder for this preprintthis version posted December 20, 2019. ; https://doi.org/10.1101/2019.12.19.882456doi: bioRxiv preprint

Page 6: Quantitative and qualitative analysis of asynchronous neural activity · 2019. 12. 19. · Quantitative and qualitative analysis of asynchronous neural activity ... via a scenario

6

with the behavior of the quenched network for J . 0.3,while increasing deviations emerge for larger couplingstrengths. Interestingly, the behavior of the annealedsetup is very close to the theoretical prediction νT [3].This is not entirely surprising, since, as anticipated, thetheoretical approach implicitly assumes an annealed con-nectivity. An additional justification for this agreement isthe Cv-values reported in Fig. 2(d) (see the dashed curve)which are much smaller than in the quenched case andcloser to 1, the value corresponding to a Poisson process.

IV. SELF-CONSISTENCY

In the previous section we have seen that quenchedand annealed networks behave in a substantially differentway, when the coupling strength is larger than J = 0.25.To what extent is this difference the signature of the cru-cial role played by a fixed structure of the synaptic con-nections?

In this section we address this issue by implement-ing a self-consistent approach, where the input currentis assumed to be the superposition of independent sig-nals, each sharing the same “statistical” properties ofthe single-neuron activity. Two different approximationschemes are hereby discussed: (i) the hypothesis of a per-fectly renewal process (RP); (ii) mutually uncorrelatedfrequency channels (also termed Gaussian approxima-tion). Here below we show that the former one providesa more accurate representation of the neural activity.

A. Renewal process

A renewal process is fully characterized by the ISIprobability distribution Q(T ). Assuming Q(T ) is known,a typical spike sequence can be readily generated by ran-domly drawing a series of T values accordingly to thisdistribution. At variance with Ref. [11], where the au-thors suggested the idea of approximating the synapticcurrent with a renewal process, here we limit ourselvesto assume that the single-neuron output activity is anRP, but we do not extend the assumption to the input,which is treated as the superposition of K independentRPs. This is an important difference since, as alreadyremarked in [21], the superposition of RPs is not renewalitself unless the single processes are purely Poissonian(this is not our case). So, at variance with Ref. [11], werelax the condition of a strictly renewal input process andreplace this Ansatz with the more general hypothesis ofa superposition of independent RPs.

In practice, we have implemented the following recur-sive procedure: given the ISI distribution Qk(T ) deter-mined in the kth step, we have generated the synapticcurrent RI of a generic neuron (in the (k+ 1)st step) bysuperposingK independent RPs all built according to thesame distribution Qk(T ) (under the constraint that bKspikes are excitatory and the remaining ones inhibitory).

0 10 20 30 50 60 70ν [Hz]0

1

2

3

4

5

6C

v

48 49 50

ν [Hz]3.85

3.90

3.95 Cv

70 72 74ν [Hz]

4.85

4.9

Cv

Pa

Pq

Pq

Pr

Ps

Pr

Ps

FIG. 5. Results of the recursive procedure based on the re-newal process and the Gaussian approximation. Pq and Paidentify the dynamics of quenched and annealed networks,respectively. Red circles (green diamonds) refer to the it-erates of the RP iterative procedure while starting from Pa(Pq); green diamonds are better visible in the enlargementreported in the upper inset, where Pr denotes the fixed pointof the renewal process approximation. Blue squares refer tothe iterative process based on the Gaussian approximation:Pq is the initial condition, while Ps denotes the fixed point ofthis approach (see the lower inset for an enlargement of thelater stages). All data have been obtained for J = 0.8).

Upon afterwards integrating the single-neuron equation,we have generated the firing activity induced by the cur-rent RI, thereby determining the (k + 1)st distributionQk+1(T ).

We first focused on J = 0.8, since the theoretical pre-diction νT [6] is significantly inaccurate for this couplingstrength. The initial condition Q0(T ) has been selectedas the distribution generated by a quenched network ofN = 105 neurons with an in-degree K = 1, 000. Thecorresponding firing rate is 〈ν〉 = 50.4 Hz and its co-efficient of variation is 〈Cv〉 = 3.97 [22]. This pair ofvalues is represented by the point Pq in Fig. 5: it corre-sponds to the projection of the asynchronous state of thequenched network in this two-dimensional space. Theiterates of the recursive procedure have been projectedon the same plane; they are so close to each other to behardly discernible in the main panel (see the enlargedplot presented in the upper inset of Fig. 5 for a clearerrepresentation). The closeness among consecutive iter-ates is confirmed by the shape of the ISI distribution: inFig. 3(c), we see that Q1(T ) is practically indistinguish-able from Q0(T ). Altogether, these observations stronglyhint at the existence of a fixed point of the RP recursiveprocedure in the vicinity of Pq. Further iterates start sep-arating from each other, suggesting that the fixed point isa saddle, which initially attracts the trajectory along thestable manifold and eventually drives it away along theunstable manifold. If, in analogy to what done in Ref. [12]

.CC-BY-NC-ND 4.0 International licenseavailable under a(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made

The copyright holder for this preprintthis version posted December 20, 2019. ; https://doi.org/10.1101/2019.12.19.882456doi: bioRxiv preprint

Page 7: Quantitative and qualitative analysis of asynchronous neural activity · 2019. 12. 19. · Quantitative and qualitative analysis of asynchronous neural activity ... via a scenario

7

for the Gaussian approximation, we include memory ef-fects by building the new ISI PDF as the average of thelast two distributions, the saddle is stabilized: the result-ing fixed point is represented in Fig. 5 as Pr. The nonperfect correspondence between Pq and Pr may have adouble rationale: the RP assumption is not exact; thenetwork size used to determine Pq is not large enough.

In order to test the quality of the RP approximation,we have studied the correlations of the sequence Tn ofconsecutive ISIs, by estimating the so-called serial corre-lation coefficient [23]

C(m) =〈Tn+mTn〉 − 〈Tn〉2

〈T 2n〉 − 〈Tn〉2

. (5)

In a strictly renewal process C(m) = 0 for m ≥ 1. Testsmade on the neurons of a quenched network for N = 105

show that C(1) is at most of order 10−3, suggesting thatthe neural activity is well approximated by an RP. Onthe other hand, since the order parameter ρ is still rela-tively large for N = 105 (ρ = 0.17), finite-size affects areprobably the predominant source of differences betweenPq and Pr.

What if the same recursive procedure is applied, start-ing from the dynamical regime exhibited by the annealednetwork (see the point Pa in Fig. 5). Forward iteratesrapidly moves away from Pa and approach Pr (see thefull circles in Fig. 5). The increasing amplitude of the“transversal” fluctuations confirm that Pr is a saddlepoint. Furthermore, the recursive procedure shows thatPa – a fixed point of the annealed process – is not afixed point of the RP iterative procedure. The reasonis that while the temporal correlations exhibited by thesingle-neuron activity (encoded in the bursting activity)are preserved by the RP approximation, they are lostin the annealed setup because of the random reshufflingof the synaptic connections. The separation between Paand Pr implicitly suggests the important role played bythe bursting activity as it will be confirmed in the follow-ing section.

Finally, we have implemented the RP approach alsofor smaller J-values, always finding evidence of a weaklyunstable fixed point (actually, the degree of instabilitydecreases upon decreasing J). The resulting message isthat the stable asynchronous dynamics exhibited by thequenched network is well reproduced by an unstable fixedpoint of a recursive transformation based on the RP ap-proximation.

B. Power spectrum

Fourier analysis offers the opportunity for additionalverification of the validity of the RP approximation. InFig. 4, we compare the power spectrum of the single-neuron activity after the first iterate obtained under theRP approximation [24] (see the purple curve) with thespectrum exhibited by the quenched network. The agree-ment is quite good for all of the three tested coupling

1 10 100 1000 f [Hz]0

0.01

0.02

D

FIG. 6. Degree of phase correlations D(f) of the Fouriertransform of the spiking activity for J = 0.8. The lowermostblue line corresponds to the RP; the uppermost black solidline corresponds to the activity of the quenched network. Fi-nally, the red line corresponds to the annealed network.

strengths, the major discrepancy being the absence ofpeaks at multiples of νd = 1818 Hz for J = 0.8, whichcannot be reproduced by the RP approximation, sincethe delay is not included in such formulation.

Next, we have estimated directly the correlationsamong the phases of the Fourier modes, by computingD(f) (see Eq. (3)) both for the quenched network and theRP approximation (see upper and lower curves in Fig. 6,respectively). Phase correlations appear to be small inboth cases (look at the vertical scale): we attribute thelarger amplitude exhibited by the quenched network tothe presence of a residual collective dynamics, absent bydefinition in the RP approximation. A comparably smalllevel of correlations is found also in the annealed network(see the almost flat red line).

As a second test of the relevance of phase-correlations,we have investigated the consequence of phase random-ization within the RP procedure. More precisely, giventhe synaptic current u(t) and its Fourier transform u(f),we have generated a new transform uN (f) = |u(f)|eiφ(f),by randomly assigning the phase φ(f) to the frequency f .A new signal uN (t) is then obtained by back transform-ing uN (f). The resulting spectrum of the firing activityof a neuron subject to the current uN (t) is presented inFig. 7 (see the red curve). The difference with the orig-inal spectrum (see the lower black curve) is not entirelynegligible: it is around around 20% in the low frequencyregion.

Finally, we have implemented the recursive procedure

proposed in Ref. [11], here briefly recalled. Let S(o)k (f)

denote the power spectrum of the single-neuron spiking

activity at the k-th recursive step. Let also S(i)k+1(f) de-

note the power spectrum of the synaptic current in the(k + 1)st recursive step. In the asynchronous regime,

.CC-BY-NC-ND 4.0 International licenseavailable under a(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made

The copyright holder for this preprintthis version posted December 20, 2019. ; https://doi.org/10.1101/2019.12.19.882456doi: bioRxiv preprint

Page 8: Quantitative and qualitative analysis of asynchronous neural activity · 2019. 12. 19. · Quantitative and qualitative analysis of asynchronous neural activity ... via a scenario

8

100

101

102

103

f [Hz]

102

103

S(f

) [H

z]

FIG. 7. Power spectra for the network activity. The blackcurve corresponds to the spectrum of the activity of a singleLIF neuron driven within the RP approach (it is basicallyindistinguishable from the true quenched network activity).The red curve is obtained by randomizing the phases of theinput signal. Data refers to J = 0.8.

the synaptic current is the superposition of K indepen-dent signals (bK excitatory and (1− b)K inhibitory), all

characterized by the same spectrum S(o)k (f). Taking into

account the amplitude of the single spikes, we have that

S(i)k+1 = [J2K(b + (1 − b)g)]S

(o)k (f) [25]. The definition

of the procedure is completed by adding the “rule” to

generate S(o)k+1, given S

(i)k+1(f). This is done by feeding a

single neuron with a phase-randomized spectrum (see theparagraph above). The self-consistent solution is finally

identified by the condition S(o)k+1 = S

(o)k .

We have implemented this approach with an Euler in-tegration step δt = 10−3ms starting from the initial con-dition Pq, the best proxy for the asynchronous regime.The first 33 iterates are reported in Fig. 5, where we seethat they move away from Pr (see the blue squares) andapproach a seemingly stable fixed point Ps. The rela-tively large difference between Ps and Pq suggests thatthis approximation scheme is not so accurate as the RPmethod and implicitly means that the phase correlationsbuilt while integrating the LIF equation are not negligi-ble [26].

V. BURSTING ACTIVITY

In the previous sections we have seen that for strongcoupling the neural activity is characterized by a largeCv, a typical indicator of burstiness. Here, we discussmore in detail the properties of this form of asynchronousregime, starting from the basic question of how it is pos-sible for it to be self-sustained.

In the asynchronous regime, the average input current

-15 -10 -5 0R<I> [mV]

0

20

40

60

80

> [H

z]

J=0.8

J=0.5

J=0.25

FIG. 8. Firing rate 〈ν〉 versus the synaptic “current” R〈I〉 fordifferent coupling strengths for the quenched network (solidline and red circles). Zero coupling corresponds to the right-most point, where the synaptic current obviously vanishes.The vertical dashed line separates the region where neuronsoperate above (right) from below (left) threshold. The greencrosses report the outcome of the annealed network. The twocrosses deviating from the solid line belong to J = 0.5 and0.8 in the annealed setup.

induced by the synaptic coupling is

〈I〉 =J

R(nE − gnI) =

KJ

Rν(b(g + 1)− g))

where nE (nI) denotes the number of excitatory (in-hibitory) spikes received per time unit. Dependingwhether R0〈I〉 is larger or smaller than Vth −RI0 = −4,the neuron operates either above or below threshold. Infact, in the latter case, the velocity field crosses the zeroaxis below the threshold Vth, preventing threshold pass-ing. In Fig. 8, we plot the firing rate versus 〈I〉 for dif-ferent values of the coupling strength J (see the solidline): increasing J corresponds to moving leftward alongthe curve, starting from the rightmost point, which cor-responds to the uncoupled limit. Upon increasing J ,〈I〉 decreases monotonically: this is the consequence ofthe prevalent inhibition. At the same time, the firingrate, after an initial drop, starts growing; this happensfor J ≈ 0.25, as it can be inferred by comparing withFig. 2(a). The increase continues also when the neuronoperates below threshold and surpasses the activity ofthe uncoupled regime.

In the same figure, we report also the outcome ofannealed-network dynamics (see the crosses): for smallR〈I〉, i.e. for small coupling we see an almost perfectcoincidence. On the other hand, by further decreasingthe internal current (i.e. upon increasing the couplingstrength), the firing rate does not increase in the annealednetwork, confirming the qualitatively different behaviorexhibited by the two setups. The annealed network op-erates above threshold.

.CC-BY-NC-ND 4.0 International licenseavailable under a(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made

The copyright holder for this preprintthis version posted December 20, 2019. ; https://doi.org/10.1101/2019.12.19.882456doi: bioRxiv preprint

Page 9: Quantitative and qualitative analysis of asynchronous neural activity · 2019. 12. 19. · Quantitative and qualitative analysis of asynchronous neural activity ... via a scenario

9

-300 -200 -100 0 Vthv [mV]

0.76

0.8

0.84

sE

-50 -40 -30 -20 0 10 20v [mV]0

20

40

60

Fef [

mV

](a)

(b)

FIG. 9. (a) Fraction sE of excitatory spikes versus the currentvalue V for J = 0.8. The horizontal line corresponds to theaverage global fraction b = 0.8. (b) Effective velocity fieldFef as from Eq. (6).

A. Correlations between membrane potential andsynaptic current

The counter-intuitive activity displayed by thequenched network requires an explanation. We have ver-ified that the effective self-induced excitation is not theresult of a symmetry breaking: all neurons (both exci-tatory and inhibitory) behave in the same way, as theyshould. More instructive information can be extracted byexploring the correlations between the actual value of themembrane potential and the quality (excitatory vs. in-hibitory) of the spike received by a given neuron. In otherwords, we have computed the relative fraction sE(V )dVof excitatory spikes received when V ∈ [V, V +dV ]. If thereceiving times were uncorrelated with the membrane po-tential, then sE(V ) would be independent of V and equalto b. Actually, this is expected within the framework ofa δ-correlated input signal as assumed in [3].

Instead, in Fig. 9(a), we see sizeable deviations, es-pecially in the vicinity of Vth, where sE is significantlylarger than b, hinting at a higher excitation than a prioriforeseeable. We have verified that, as expected, the av-erage of sE – computed along the V -axis and weightedaccording to the stationary distribution P (V )) – is equalto b - see the horizontal line.

A perhaps more enlightening representation of therole played by the V -dependence of sE is proposed inFig. 9(b), where we plot the effective velocity field

Fef = RI0 − V +KJτ〈ν〉(sE(1 + g)− g) (6)

where the V -dependent sE replaces b. Interestingly, theeffective velocity field does not cross the zero axis belowthreshold, showing that the neuron effectively operatesabove threshold, in spite of R〈I〉 < Vth−RI0. So, we canconclude that including the V -dependence of sE into the

neural dynamics helps solving the paradox of a neuronoperating on average below threshold. On a more quan-titative level, we can interpret Eq. (6) as a deterministicevolution equation and thereby compute the firing rate νaas the inverse of the time needed to reach the thresholdVth, while starting from Vr (augmented by the refractorytime). By inserting ν- and sE(V )-values obtained fromthe network simulations for J = 0.8, we find νa = 77 Hz,to be compared with the observed rate ν = 50 Hz. Theagreement is not as good as one might had hoped for,but it should also be noted that Eq. (6) does not accountfor the (strong) input fluctuations!

Moreover, the V -dependence of sE still needs to be un-derstood. Some light can be shed by arguing as follows.Let us introduce the joint probability P (E,L) that anexcitatory spike reaches the neuron, when its membranepotential V ∈ L = [V0, Vth], where V0 is selected as thepoint where sE = b, The standard Bayesian inferencerule implies that

P (E|L) =P (L|E)

P (L)P (E)

where P (A|B) denotes the probability of observing A,given B; moreover P (E) = b, while P (L) is the proba-bility of V > V0, and P (E|L) is just the average of sEover L. Let us now focus on P (L|E)/P (L): this is theprobability of V > V0 when an excitatory spike arrives(rescaled to the unconditional probability to stay in L).If excitatory spikes arrive in bursts, for many of them thecorresponding V -value is relatively large as a consequenceof the excitation provided by the previous spikes. There-fore, it is natural to expect P (L|E)/P (L) > 1. This isprecisely what we see in Fig. 9, where one can notice thatsE is larger than b close to threshold. Consistency thenimposes that sE < b further away.

B. Synaptic current: an Ornstein-Uhlenbeckprocess

The role of correlations can be analysed from a dif-ferent point of view: since the neuron is typically underthe action of a negative current, its membrane potentialis kept away from threshold (V < Vth). Only when rel-atively strong positive fluctuations of the input currentarise, the neuron can overcome the threshold and emita spike. If the correlations are sufficiently long-lasting,the fluctuation may stand long enough to allow for theemission of a sequence of spikes and give rise to a “burst”.This mechanism has been already investigated in the pastto quantify the spiking activity of a sub-threshold neu-ron subject to Ornstein-Uhlenbeck (OU) noise, findingthat a long correlation time gives rise to a bursting activ-ity [23, 27]. Unfortunately, we cannot make use of theirformulas, since the correlation time is not much longerthan τ . We can, nevertheless, proceed in a purely nu-merical way by approximating the input current I with

.CC-BY-NC-ND 4.0 International licenseavailable under a(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made

The copyright holder for this preprintthis version posted December 20, 2019. ; https://doi.org/10.1101/2019.12.19.882456doi: bioRxiv preprint

Page 10: Quantitative and qualitative analysis of asynchronous neural activity · 2019. 12. 19. · Quantitative and qualitative analysis of asynchronous neural activity ... via a scenario

10

the outcome of an OU stochastic equation, namely

τcI = 〈I〉 − I + ξ ,

where 〈I〉 is the average current observed in the numer-ical simulations of the quenched network, while τc is in-put correlation time and finally ξ is a δ-correlated whitenoise (〈ξ(t′)ξ(t′ + t)〉 = σ2δ(t). We have thereby tunedτc and σ2, until the neuron activity is characterized bythe expected firing rate and the corresponding Cv. ForJ = 0.8, we have found τc ≈ 160 ms and σ2 ≈ 6.2 · 10−2.As a bonus, the resulting ISI distribution turns out tobe quite similar to the expected one, the major differ-ence being the peak which instead of being located inT = 1ms (see the inset in Fig. 3(c)), occurs for T ≈ 4ms. Altogether, one can nevertheless conclude that theOU approximation provides a reasonable description ofthe input current.

We have implemented the same procedure for J = 0.5:in spite of the similar bursting activity, the neuron oper-ates on average above threshold and we have not foundany way to parametrize the OU process so as to repro-duce the observed activity. On the other hand, a goodreproduction of the neural activity is found for J = 1,by assuming τc = 145 and σ2 = 0.28. Two interest-ing comments are in order: (i) upon increasing the cou-pling, the correlation time does not increase: it seemsthat τC ≈ 140 − 160 is an intrinsic property of the net-work; (ii) the noise amplitude increases by more than afactor 4, while passing from J = 0.8 to J = 1 and thisis the reason why the firing rate is larger in the lattercase, even though the neuron operates much more belowthreshold. The increase of the effective noise can be at-tributed to two factors: a minor contribution comes fromthe increased coupling strength (from 0.8 to 1); a morerelevant contribution is the increased fluctuations of thesingle-neuron activity quantified by the Cv.

C. Synaptic current: symbolic correlations

We conclude this section by looking at correlationsfrom a different point of view. As shown in Sec. IV, theoutput activity of the single neuron is well approximatedby an RP, but we do not expect the same to be true forthe input, obtained by superposing K independent suchprocesses. In order to investigate the way correlationsmanifest themselves, we separately computed the ISI dis-tribution of all excitatory and inhibitory spikes receivedby a given neuron.

In Fig. 10 we report the ISI distribution of excitatoryspikes (inhibitory spikes follow the same statistics) forboth the original quenched network (black curve) andwithin the RP approximation (red curve). T representsthe average separation between consecutive spikes, i.e. Tis equal to the average single-neuron ISI divided by 800– the total number of incoming excitatory synaptic con-nections. The red curve follows a clean Poissonian dis-tribution, while the quenched network exhibits a slower

0 2 4 6 10 12 14T/T10

-5

10-4

10-3

10-2

10-1

100

Q(T

)

FIG. 10. The effective distribution Q(T ) of inter-excitatory-spikes received by a single neuron for J = 0.8. The black curverefers to the quenched network, the red one to the renewalprocess approximation. The horizontal axis is rescaled by themean firing rate of the respective population of excitatoryspikes.

than exponential decay (in this time range); furthermore,in the latter case, the first channel is very large becauseof the unavoidable presence of avalanches occurring inthe quenched setup (see [8]). We attribute most of thedeviations from a pure exponential to the residual pres-ence of collective dynamics. In any case, this discrepancyis a minor issue: the relevant correlations are those be-tween excitatory and inhibitory spikes, as revealed by thefollowing test. We have fed a single neuron with two dif-ferent signals: (i) a perfectly Poisson process composedof independent excitatory and inhibitory spikes; (ii) asynthetic signal built by assuming a Poisson distributionof consecutive spikes with the same rate as the quenchednetwork, but keeping the original symbolic ordering (i.e.EEIEEEIE . . ., where the letter E/I means that thespike is either excitatory or inhibitory) observed in thequenched network.

The resulting membrane-potential distributions of theneuron are reported in Fig. 11. The black curve, ob-tained by using the above mentioned synthetic signal, isvery similar to the original distribution. On a quantita-tive level, the resulting firing rate, the Cv and the PDFof the ISIs are very close to the values exhibited by theRP approximation (deviations are smaller than 1%): seethe inset of panel (c) in Fig. 3. On the other hand, thered curve, originating from the strictly Poisson processis shifted towards very negative v-values and nearly van-ishes close to the threshold, suggesting a very low firingactivity as indeed observed.

Altogether this proves that the relevant correlationsare contained in the symbolic representation of the spikesequence.

.CC-BY-NC-ND 4.0 International licenseavailable under a(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made

The copyright holder for this preprintthis version posted December 20, 2019. ; https://doi.org/10.1101/2019.12.19.882456doi: bioRxiv preprint

Page 11: Quantitative and qualitative analysis of asynchronous neural activity · 2019. 12. 19. · Quantitative and qualitative analysis of asynchronous neural activity ... via a scenario

11

-200 -100 0v [mV]0

0.005

0.01

P(v)

FIG. 11. Probability density of the membrane potential P (v).The black curve exhibiting the divergence in v = 10 corre-sponds to the synthetic signal described in text; the red curveis obtained by feeding the neuron with a purely Poisson pro-cess with the same firing rate.

VI. CONCLUSIONS AND OPEN PROBLEMS

In this paper we have shown that upon increasing thecoupling strength J (and for J > 0.25), a slightly in-hibitory sparse network of LIF neurons operates increas-ingly below threshold and yet fires at an increasingly fastrate. This claim is supported by careful numerical simu-lations, tailored so as to marginalize the effects of collec-tive synchronization.

This counter-intuitive, self-sustained, activity observedin quenched networks, disappears in annealed networks,i.e. in setups where the synaptic connections are con-tinuously randomly reshuffled. In the latter case, theneural activity is both weaker and more homogeneous(for J = 0.8, the firing rate drops by a factor about 4).The difference between quenched and annealed setup isreminiscent of replica symmetry breaking [28], but theanomaly of the phenomenon is mitigated by the obser-vation that the quenched-network dynamics can be re-produced to a high degree of accuracy by an approach,the renewal-process (RP) approximation, which does nottake into account the structure of the synaptic connec-tions. Still, the comparison between quenched and an-nealed dynamics (see Fig. 8 for the most enlighteningrepresentation)) seems to suggest the presence of a phasetransition when J is increased. It looks like the tworegimes deviate from one another above J ≈ 0.25. Thisis reminiscent of the claim made by Ostojic about theexistence of two distinct asynchronous regimes [4]. Thisclaim has been criticised in Ref. [13]; we are also un-able to find evidence of a qualitative difference betweenthe two regimes (above and below a supposedly criticalpoint Jc).

Within the RP approximation, the neural activity isfully characterized by the ISI distribution. In the limitof large coupling strengths, such a distribution exhibits

0 0.2 0.4 0.6 0.8J

0

20

40

> [

Hz]

0 0.2 0.4 0.6 0.8J

0

1

2

3

4 <Cv>(a) (b)

FIG. 12. Neural activity for different delay times. The solidcurves all refer to τd = 0.55, while crosses refer to τd = 0.3.All simulations are performed in a network with N = 105

neurons and a connectivity K = 1000.

a power-law decay, similar to what found while studyingthe response of a single neuron to Ornstein-Uhlenbeckprocesses [23, 27] and similar to experimental observa-tions made in the sensorimotor cortex of rats performingbehavioural tasks [29]. It should, however, be noticedthat in our case, the scaling range is much smaller thanin the experimental observations.

The recursive process based on the RP approximationproves rather accurate in spite of not taking into accountthe delay. It is therefore natural to ask whether thisis true also in the quenched network. Simulations per-formed for different delay values confirm a substantialindependence of the outcome on τd. In Fig. 12, we com-pare the firing rate and the Cv obtained for τd = 0.3 withthe original simulations (performed for τd = 0.55).

Finally, we wish to comment on the peculiar behav-ior of the network observed for large coupling strengths.The strong firing activity is self-sustained by its bursti-ness (signalled by the large Cv values), which, de facto,provides the relatively long correlations required to leta neuron below threshold fire. This clarifies the reasonwhy the theoretical formula based on the assumption ofδ-correlated current fluctuations, fails to reproduce thisregime. Recently, a more sophisticated self-consistentapproach has been developed, where the Fokker-Planckequation has been augmented to account for temporalcorrelations in the synaptic current [30]. Its (numerical)implementation to a weak-bursting regime looks promis-ing. It will be worth exploring its validity in a moreinhibition dominated regime such as the one explored inthis paper. Interestingly, the bursting activity is repro-duced also assuming a strictly Poisson ISI distribution,but retaining the correlations contained in the symbolicrepresentation of the spike types (i.e. excitatory vs. in-hibitory). A simple quantification of such correlationsmight open yet another route for a quantitative charac-terization of the neural activity.

ACKNOWLEDGMENTS

A.T. received financial support by the Excellence Ini-tiative I-Site Paris Seine (Grant No ANR-16-IDEX-008), by the Labex MME-DII (Grant No ANR-11-LBX-

.CC-BY-NC-ND 4.0 International licenseavailable under a(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made

The copyright holder for this preprintthis version posted December 20, 2019. ; https://doi.org/10.1101/2019.12.19.882456doi: bioRxiv preprint

Page 12: Quantitative and qualitative analysis of asynchronous neural activity · 2019. 12. 19. · Quantitative and qualitative analysis of asynchronous neural activity ... via a scenario

12

0023-01) (together with A.P. and E.U.) and by theANR Project ERMUNDY (Grant No ANR-18-CE37-

0014), all part of the French programme “Investissementsd’Avenir”.

[1] A. S. Ecker, P. Berens, G. A. Keliris, M. Bethge, N. K.Logothetis, and A. S. Tolias, science 327, 584 (2010).

[2] A. Renart, J. de la Rocha, P. Bartho, L. Hollender,N. Parga, A. Reyes, and K. D. Harris, Science 327, 587(2010).

[3] N. Brunel, Journal of Computational Neuroscience 8, 183(2000).

[4] S. Ostojic, Nat Neurosci 17, 594 (2014).[5] C. van Vreeswijk and H. Sompolinsky, Science 274, 1724

(1996).[6] R. Capocelli and L. Ricciardi, Kybernetik 8, 214 (1971).[7] E. Ullner, A. Politi, and A. Torcini, Chaos: An Interdis-

ciplinary Journal of Nonlinear Science 28, 081106 (2018),https://doi.org/10.1063/1.5049902.

[8] A. Politi, E. Ullner, and A. Torcini, The European Phys-ical Journal Special Topics 227, 1185 (2018).

[9] M. J. E. Richardson and R. Swarbrick, Phys. Rev. Lett.105, 178102 (2010).

[10] S. Olmi, D. Angulo-Garcia, A. Imparato, and A. Torcini,Scientific reports 7, 1577 (2017).

[11] B. Dummer, S. Wieland, and B. Lindner, Frontiers incomputational neuroscience 8 (2014).

[12] R. F. Pena, S. Vellmer, D. Bernardi, A. C. Roque, andB. Lindner, Frontiers in computational neuroscience 9(2018).

[13] R. Engelken, F. Farkhooi, D. Hansel, C. vanVreeswijk, and F. Wolf, F1000Research 5 (2016),10.12688/f1000research.9144.1.

[14] E. Ullner and A. Politi, Physical Review X 6, 011015(2016).

[15] A. Politi, A. Pikovsky, and E. Ullner, The EuropeanPhysical Journal Special Topics 226, 1791 (2017).

[16] W. Gerstner, W. M. Kistler, R. Naud, and L. Paninski,Neuronal Dynamics: From Single Neurons To NetworksAnd Models Of Cognition (Cambridge University Press,

Cambridge, 2014).[17] D. Golomb, Scholarpedia 2, 1347 (2007).[18] One should also remember that because of the refractory

period, T > τR.[19] S. Ostojic, Journal of neurophysiology 106, 361 (2011).[20] Here and everywhere power spectra are represented, they

are normalized in such a way that the total power isobtained by integrating over all positive frequencies: 0 ≤f < +∞.

[21] B. Lindner, Physical Review E 73, 022901 (2006).[22] In finite networks, sample-to-sample fluctuations are ex-

pected. Simulations of five different networks show thatthe standard deviation of 〈ν〉 is σν ≈ 0.4, while that of Cvis σC = 0.01. Additionally, one expects the single stepsof the recursive procedure do be affected by statisticalfluctuations: we have verified that the uncertainty of 〈ν〉is about 0.05, while that of Cv is approximately 0.005.

[23] T. Schwalger and L. Schimansky-Geier, Physical ReviewE 77, 031914 (2008).

[24] No appreciable differences can be noticed while referringto the following iterates.

[25] Leaving aside the zero-frequency channel which con-tributes to the average and is treated differently.

[26] The quantitative differences with the results for Ps re-ported in Ref. [12] are quite likely to be attributed tothe lack of accuracy in the integration scheme employedtherein.

[27] R. Moreno-Bote and N. Parga, Physical review letters92, 028102 (2004).

[28] M. Mezard, G. Parisi, and M. Virasoro, Spin GlassTheory and Beyond (WORLD SCIENTIFIC, 1986)https://www.worldscientific.com/doi/pdf/10.1142/0271.

[29] Y. Tsubo, Y. Isomura, and T. Fukai, PLoS computa-tional biology 8, e1002461 (2012).

[30] S. Vellmer and B. Lindner, Phys. Rev. Research 1, 023024(2019).

.CC-BY-NC-ND 4.0 International licenseavailable under a(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made

The copyright holder for this preprintthis version posted December 20, 2019. ; https://doi.org/10.1101/2019.12.19.882456doi: bioRxiv preprint