Abstract. arXiv:2010.08195v1 [q-bio.NC] 16 Oct 2020Abstract. In neuroscience, learning and memory are usually associated to long-term changes of connection strength between neurons

STOCHASTIC MODELS OF NEURAL SYNAPTIC PLASTICITY

PHILIPPE ROBERT AND GAETAN VIGNOUD1

Abstract. In neuroscience, learning and memory are usually associated to

long-term changes of connection strength between neurons. In this context,

synaptic plasticity refers to the set of mechanisms driving the dynamics of neu-ronal connections, called synapses and represented by a scalar value, the synap-

tic weight. A Spike-Timing Dependent Plasticity (STDP) rule is a biologically-

based model representing the time evolution of the synaptic weight as a func-tional of the past spiking activity of adjacent neurons.

If numerous models of neuronal cells have been proposed in the mathemat-

ical literature, few of them include a variable for the time-varying strengthof the connection. In this article, a new, general, mathematical framework to

study the phenomenon of synaptic plasticity associated to STDP rules is intro-duced. A system composed of two neuronal cells connected by a single synapse

is investigated and a stochastic process describing its dynamical behavior is

presented and analyzed. The notion of plasticity kernel is introduced as a keycomponent of plastic neural networks models. We show that a large number

of STDP rules from neuroscience and physics applied to neural systems can

be represented by this formalism.Experiments show that long-term synaptic plasticity evolves on a much

slower timescale than the cellular mechanisms driving the activity of neuronal

cells. For this reason a scaling model of our stochastic model is also introducedand averaging principles for a sub-class of plasticity kernels are stated, and

proved in a companion paper. These results are used to analyze two STDP

models widely used in applied physics: Pair-based rules and calcium-basedrules. We compare results of computational neuroscience on models of timing-

based synaptic plasticity with our results derived from averaging principles. Aclass of discrete models of STDP rules is also introduced and studied for the

analytical tractability of its solutions in the light of averaging principles.

Contents

1. Introduction 22. Models of Neural Plasticity 73. Markovian Plasticity Kernels 214. A Scaling Approach 245. Pair-Based Rules 276. Calcium-Based Rules 317. Discrete Models of Calcium-Based Rules 34References 38Appendix A. Discussion of Some Aspects of STDP Models 42Appendix B. Graphical Representation of Models of Plasticity 46Appendix C. Operators of Fast Systems of STDP models 52

Date: December 31, 2020.1Supported by PhD grant of Ecole Normale Superieure, ENS-PSL.

1

arX

iv:2

010.

0819

5v1

[m

ath.

PR]

16

Oct

202

0

2 PH. ROBERT AND G. VIGNOUD

1. Introduction

Central nervous systems, as the brain, are the main substrate for memory andlearning, two essential concepts in the understanding of behavior.

It is widely accepted that neurons constitute the main relay for informationin complex neural networks composing the mammalian and insect brains. Thismulti-scale system, ranging from single neuronal cells to complex brain areas, isknown to be the basis of memory consolidation that achieves the transformationof a temporary information into a long-lasting stable memory. The memory trace,or engram, has been the focus of studies in neuroscience, see Tonegawa et al. [55]for example. Biological, computational and mathematical models are developed tounderstand mechanisms by which an engram emerges during learning, maintainsitself, and evolves with time.

The encoding of memory is assumed to be represented by the strengths of theconnections, also known as synapses, between neurons. They are the key com-ponents for the transmission of information between connected neurons. From abiological point of view, a synapse is a structure between two neurons that cantransmit chemical/electrical signals. A neuronal connection is unilateral in thesense that the signal goes from an input neuron, called the pre-synaptic neuron, tothe output one, the post-synaptic neuron. The intensity of the connection is referredto as the synaptic efficacy/strength, it will be represented by a scalar variable, thesynaptic weight. The usual convention is to describe the impact of an input signalfrom a pre-synaptic neuron, a spike, as a jump of the membrane potential of thepost-synaptic neuron. The amplitude of the jump is used to quantify the synapticweight.

A synaptic plasticity mechanism is defined as a collection of activity-dependentcellular processes that modifies the synaptic efficacy of a population of synapses.During learning, specific patterns of neural activity may elicit short, from millisec-onds to seconds, and/or long, from minutes to hours, term changes in the associ-ated synaptic weights. In this context, memory is directly associated to synaptic,see Takeuchi et al. [54].

1.1. The State of a Neuronal Cell. In neuroscience numerous models of an in-dividual neuronal cell and neuronal networks have been used to investigate learningabilities and plasticity. See Gerstner et al. [19] for a review.

The state of a neuron is usually represented by its membrane potential X which isa key parameter to describe the cell activity. The leaky-integrate-and-fire model de-scribes the time evolution of the membrane potential as a resistor-capacitor circuitwith a constant leaking mechanism. Due to external input signals, the membranepotential of a neuron may rise until it reaches some threshold after which a spike isemitted and transferred to the synapses of neighboring cells. A large class of neu-ral models based on this leaky-integrate-and-fire hypothesis has been developed,see Gerstner et al. [19] and references within.

To take into account the important fluctuations within cells, due to the spikingactivity and thermal noise in particular, a random component in the cell dynamicshas to be included in mathematical models describing the membrane potential evo-lution. For several models this random component is represented as an independentadditive diffusion component, like Brownian motion, of the membrane potential.

STOCHASTIC MODELS OF NEURAL SYNAPTIC PLASTICITY 3

In our approach, the random component is added at the level of the generation ofspikes. When the value of the membrane potential of the output neuron is at X=x,a spike occurs at rate β(x) where β is the activation function. See Chichilnisky[7] for a discussion. In particular the instants when the output neuron spikes arerepresented by an inhomogeneous Poisson process. When the synaptic weight of aconnection between a pre-synaptic neuron and a post-synaptic neuron is fixed andequal to W , the time evolution of the post-synaptic membrane potential (X(t)) isrepresented by the following stochastic differential equation (SDE):

(1) dX(t) = −1

τX(t) dt+WNλ(dt)− g (X(t−))Nβ,X(dt),

where X(t−) is the left limit of X at t>0,

— τ is the exponential decay time constant of the membrane potential associ-ated to the leaking mechanism;

— the sequence of firing instants of the pre-synaptic neuron are assumed to bea Poisson point process Nλ on R+ with rate λ. At each pre-synaptic spike,the membrane potential X is increased by the amount W . If W>0 thesynapse is said to be excitatory, whereas for W<0 the synapse is inhibitory;

— the sequence of firing instants of the post-synaptic neuron is an inhomo-geneous Poisson point process Nβ,X on R+ whose intensity function ist 7→β(X(t−)). See Kingman [31].

— The drop of potential due to a post-synaptic spike is represented by thefunction g. When the post-synaptic neuron fires in state X(t−)=x, itsstate X(t) just after the spike is x−g(x).

Considering that the point process Nβ,X depends on (X(t)), Relation (1) can beseen as a fixed point equation.

1.2. Synaptic plasticity. Synaptic plasticity is associated to the time evolution ofthe synaptic weight, i.e. when the variable W depends on time. Although synapticplasticity is a complex mechanism, some general principles have been inferred fromexperimental data and previous modeling studies. One of the founding principlesbeing Hebb’s postulate (1949):

“When an axon of cell A is near enough to excite cell B and re-peatedly or persistently takes part in firing it, some growth processor metabolic change takes place in one or both cells such that A’sefficiency, as one of the cells firing B, is increased.”

This postulate has been later on summarized by Shatz [52] as, “Cells that firetogether wire together”.

Potentiation, resp. depression, is associated to an increase, resp. a decrease,of the synaptic weight amplitude. Plasticity is described as a set of mechanismscontrolling the potentiation and the depression of synapses. It is usually expressedas depending on the pre-synaptic and post-synaptic signaling, i.e. of past instantsof pre-synaptic and post-synaptic spikes. In the literature this is referred to asSpike-Timing Dependent Plasticity (STDP). Several experimental protocols havebeen developed to elicit STDP at synapses: sequences of spikes pairing from ei-ther side of the synapse are presented to a specific synapse, at a certain frequencyand with a certain delay. As a direct consequence, the formation of memory andlearning is expressed as the outcome of some activity-dependent synaptic plastic-ity. Occurrence, magnitude and polarity of STDP have been shown to depend on


protocols used in experiments: frequency, number of pairings, types of synapseswhere it is applied, the neuronal sub-population, brain area, just to cite a few keyparameters, see Feldman [14].

We now introduce two important classes of synaptic plasticity mechanisms. Mostmodels of the literature belong to, or are a variation of, one of these two classes.

(a) Pair-based models.For these models, each pair t=(tpre, tpost) of instants of pre-synaptic andpost-synaptic spikes is associated to an increment ∆W of the synapticweight at time max(tpre, tpost),

(2) ∆W = Φ(∆t),

where ∆tdef.= tpost−tpre and Φ is the some function on R, the STDP curve.

The function Φ, usually taken from experimental data, is sharply decreasingto 0 as ∆t goes to infinity, so that pairs of distant spikes have a negligiblecontribution.

Hebbian STDP plasticity is said to occur when— a pre-post pairing, i.e. tpre<tpost leads to potentiation, ∆W>0;— a post-pre pairing, tpost<tpre, leads to depression, ∆W<0.

Experiments have shown that this type of plasticity occurs for several popu-lations of neuronal cells. See Bi and Poo [4] for example. Pair-based STDPis widely used in the literature and has served as a basis in the STDP studiesin many neural networks models.

(b) Early models can be found in Rossum et al. [49] and Rubin et al. [50].See Morrison et al. [37] for a review.

Anti-Hebbian STDP models give an opposite long-term plasticity model ofHebbian STDP models: Pre-post pairings lead to depression, and post-prepairings lead to potentiation. It has also been observed experimentally inthe striatum, see Fino et al. [15] for example.

Many variants and extensions of pair-based models have been developedover the years to fit with experimental results. Triplets-rules, described inSections 2.4.3 and 2.4.4, add a dependency between spikes of the sameneuronal cell. Additional examples can be found in Babadi and Abbott [3].

(c) Calcium-based models.Another class of models infers from explicit biological mechanisms the shapeof the STDP (that is taken as a parameter in pair-based models). A classicaland minimal example of such model uses the fact that post-synaptic calciumtraces have been found experimentally to be critical in the establishment ofplasticity, see Feldman [14] and references therein. Voltage-gated calciumchannels act as coincidence detectors for pre-synaptic and post-synapticaction potentials and enables, along with other calcium channels, the drivingof STDP by calcium currents. When the calcium concentration Cca in thepost-synaptic neuron reaches some specific threshold, STDP can be inducedaccordingly. The analogue of Relation (2) for calcium-based STDP rules is,

(3) dW (t) = F (Cca(t)) dt,

for some function F . The dynamics of Cca is only driven by instants of pre-and post-synaptic spikes. Consequently, the dependence of plasticity on


the instants of spikes is not expressed directly as in pair-based models, butthrough some intermediate biological variable. Several biophysical modelsare based on this calcium hypothesis, see Graupner and Brunel [22] for areview on calcium-based STDP rules.

It should be noted that there are other STDP models, such as the ones basedon exponential filtered traces of the membrane potential, see Clopath and Gerstner[9]. Pair-based and calcium-based models are nevertheless the most widely usedSTDP rules in large-scale plastic neural networks.

1.3. Multiple Timescales. An important feature of long-term neural plasticityis that there are essentially two different timescales in action. The membranepotential of neurons, i.e. the spiking activity and the coincidence detection, eitherthrough calcium-based mechanisms, or pair-based rules, are “fast” processes. Atypical order of magnitude is the millisecond, or tens of milliseconds at most. Thesynaptic weights on the other hand evolve more slowly. Experimental protocolsshow that significant synaptic changes appear several seconds, or even minutes,after the plasticity protocol.

Computational models of plasticity incorporate this timescale difference by onlyimplementing small updates of the synaptic weights. However, it does not reallytake into account the fact that significant changes occur after the end of the proto-col. To take into account this phenomenon, a possible approach consists in updatingthe synaptic weights with a fixed, or random, delay. This is not completely satis-factory since the evolution of the synaptic weight is generally believed to be as anintegrative process of past events rather than a delayed action. Another approach,which we will use, consists in implementing this delay through an exponentiallyfiltered process to represent the accumulation of past information.

Two different times scales are used in the model, one, fast, for neural mechanismsof the cell and another one, slow, for long-term plasticity, i.e. for the updatesof the synaptic weight. The terms Long-Term Potentiation (LTP) or Long-TermPotentiation Depression (LTD) are sometimes used to stress these differences oftimescale.

It should be noted here that fast synaptic plasticity processes exist. This isreferred to as short-term synaptic plasticity. See Zucker and Regehr [58]. We donot consider the influence of short-term plasticity in this paper.

1.4. Models of Plasticity in the Literature. To understand how synaptic plas-ticity may shape the brain, the study of STDP in neural networks has attracted alot of interest in different domains:

(a) Experiments, with measurements of a large variety of STDP rules;(b) Computational models, for numerical simulations of these protocols with

several populations of neuronal cells;(c) Mathematical models, to investigate the qualitative properties of some

generic STDP rules.

Many computational models have been developed to investigate STDP rules indifferent contexts. A detailed list of neuronal models with STDP rules would be toolong to draw, we just refer to Morrison et al. [36] for their study on balanced net-works and Ravid Tannenbaum and Burak [43] on topological properties of recurrentnetworks with STDP.


Numerous works in physics have investigated mathematical models of plasticity.We quickly review some of them. Most studies focus on the dynamics of a collectionof synaptic weights projecting to a single post-synaptic cell. There are basicallytwo types of approximations used.

(a) Separation of Timescales.The approximations done in these studies assume that there is a separationof timescales in the evolution equations. This approach is described inSection 4. The cellular processes are averaged to give a simpler dynamicalsystem for the evolution of the synaptic weight. This is a classical approachin the literature. See Kempter et al. [30], Rubin et al. [51], and Eurichet al. [13]. Akil et al. [2] uses an analogous description of the evolution ofsynaptic weights in the context of a mean-field approximation of differentpopulation of neurons.

(b) Fokker-Planck Approach.In this case, the time evolution of the synaptic strength alone is assumed tofollow a diffusion process and, consequently, has the Markovian property.The analysis is done with the associated Fokker-Planck equations and thecorresponding equilibrium distribution when it exists. See Rubin et al. [50],Horn et al. [28], Kistler and Hemmen [32], and Rossum et al. [49]. Anextension, the Kramers-Moyal expansion, see Pawula [40], is also used inthis context for some non-Markovian models.

Mathematical studies of models of plasticity are quite scarce. Most models arecentered on evolution equations of neural networks with a fixed synaptic weight.See Sections 1 and 2 of Robert and Touboul [45] for a review. In Abbassian etal. [1] and Perthame et al. [41], an ODE/PDE approach for a population of leakyintegrate-and-fire neurons is presented for a STDP rule related to the nearest neigh-bor symmetric model of Section 1b. See Chevallier et al. [6] for the connectionbetween stochastic models and PDE models. Helson [26] investigates, via an aver-aging principle, a Markovian model of a Nearest Neighbor Symmetric Model STDPrule. This is one of the few stochastic analyses in this domain.

1.5. Contributions. A mathematical model of plasticity describing a pre- and apost-synaptic neuron should include the spiking mechanisms of the two neuronalcells. It is given by the time evolution of the membrane potential X of the post-synaptic cell, as described by Equation (1). It must also include the dynamics ofplasticity of the type (2) or (3) for the time evolution of the synaptic weight W .

The difficulty lies in the complex dependence of the evolution of W with respectto the instants of spikes of both cells, the processes Nλ and Nβ,X of Equation (1).For pair-based models for example, this is a functional of all pairs of instants of bothprocesses. In general, there does not exist a simple Markovian model to describethe membrane potential dynamics and the evolution of the synaptic weight.

In Section 2, we introduce the notion of plasticity kernel which describes in ageneral way how the spiking activity is taken into account in the temporal evolu-tion of the synaptic weight as a functional of the point processes Nλ and Nβ,X .A differential system associated to the dynamics of the variables X and W is pre-sented. Under mild conditions, it is proved that it has a unique solution for a giveninitial state. It is, to the best of our knowledge, the first attempt to have a generalmathematical framework that describes most STDP rules of the literature. A large


set of examples is presented in Section 2.4, it is shown that most of STDP modelsof Morrison et al. [37], Graupner and Brunel [22], Clopath et al. [8], and Babadiand Abbott [3] can be represented within this formalism.

Another, related, difficulty is to take into account the two different timescalesmentioned above. This is done by assuming that the membrane potential X isa “fast” variable, i.e. that it evolves on a fast timescale. A scaled version of ourstochastic model is introduced in Section 4. An averaging principle for the processof the synaptic weight for this plastic neural system has to be established in thiscontext. See Papanicolalou et al. [39] and Freidlin and Wentzell [16] for a generalpresentation.

Section 3 is devoted to an important sub-class of STDP rules, Markovian plas-ticity kernels. These kernels have a representation in terms of finite dimensionalvectors whose coordinates are shot-noise processes. Calcium-based rules and pair-based rules when the STDP curve Φ is exponential have Markovian plasticity ker-nels. If a classical Markovian analysis of the associated stochastic processes is notreally possible, their main advantage is that one can formulate a tractable modelwith two timescales as described by Section 1.3, when the timescale of cellulardynamics is of the order of 1/ε with ε small. Under convenient assumptions, anaveraging principle, Theorem 2, shows that the evolution equation of the synap-tic weight W converges to a deterministic dynamical system as ε goes to 0. Thistheorem is the main result of the companion paper Robert and Vignoud [46].

Sections 5 and 6 discuss the implications of averaging principles for classicalmodels of pair-based and calcium-based STDP rules. In particular, explicit resultsfor the time evolution of the synaptic weight can be obtained for several pair-basedrules. Related results of the literature in physics are discussed in Section A.2 ofAppendix. For calcium-based STDP models, the situation is more complicatedsince an explicit representation of invariant distributions of a class of Markov pro-cesses is required to express the asymptotic time evolution of the synaptic weight.For this reason, Section 7 introduces an analytically tractable discrete model ofcalcium-based STDP rules. With a scaling approach similar to that of Section 4,the dynamical system of the asymptotic evolution of the synaptic weight can beinvestigated. Section A.1 of Appendix discusses modeling issues on the incorpora-tion of plasticity: via a time-smoothing kernel, as we do in the paper, or directlywith the instantaneous information.

2. Models of Neural Plasticity

We introduce a simple, basic, but important network with two neurons connectedby one synapse. This synapse is a unidirectional connection from the input neuronto the output neuron allowing the transmission of ‘information’.

When the input, or pre-synaptic, neuron spikes, some neurotransmitters are re-leased at the level of the synapse, where they can reach the output, or post-synaptic,neuron. Following synaptic transmission, a pre-synaptic spike increments the mem-brane potential X of the output neuron by a scalar value, the synaptic weight Wof the connection. The spikes of the input, resp. output, neuron are called pre-synaptic, resp. post-synaptic, spikes.

The dynamics of neural plasticity is described in terms of the time evolution of(X(t)) and (W (t)). For t≥0,


(a) X(t)∈R is the membrane potential of the output neuron at time t. This isthe difference between the internal and the external electric potentials ofthe neuron. The dynamics of the process (X(t)) associated to the outputneuron is a classical model of neuroscience. See Gerstner et al. [19] for asurvey.

(b) W (t)∈R represents the intensity of synaptic transmission at time t, i.e.the increment of the post-synaptic membrane potential X when the inputneuron spikes at time t. The time evolution of the process (W (t)) at timet>0 depends in general on the total sample path of ((X(s),W (s)), 0≤s≤t),in an intricate way.

To take into account inhibitory mechanisms, these two variables are real-valued and,consequently, may have negative values. In the following sections, other variableswill be added to formalize the evolution equations of (X(t),W (t)).

2.1. Definitions and Notations. The sequence of pre- and post-synaptic spikesplay an important role in the study of spike-timing dependent plasticity. Mathe-matically, it is convenient to describe them in terms of point processes. See Dawson[11] for general definitions and results on point processes.

We denote by M+(Rd+) the set of positive Radon measures on Rd+, i.e. with

finite values on any compact subset of Rd+. A point measure on Rd+, d≥1, is an

integer-valued Borelian positive measure on Rd+ which is Radon. A point mea-

sure is in particular carried by a subset of Rd+ which is at most countable and

without any finite limiting point. The set of point measures on Rd+ is denoted by

Mp(Rd+)⊂M+(Rd+), it is endowed with the natural weak topology ofM+(Rd+) andits corresponding Borelian σ-field.

If m∈Mp(Rd+) and A∈B(Rd+) is a Borelian subset of Rd+, then m(A) denotes thenumber of points of m in A, i.e.

m(A) =

∫Rd+

1A(x)m(dx).

A point process on Rd+ is a probability distribution on Mp(Rd+).Two independent Poisson point processes are assumed to be defined on a filtered

probability space (Ω,F , (Ft),P),

(a) a Poisson point process Nλ on R+ with rate λ>0, (tpre,n) is the increasingsequence of its jumps, i.e.

Nλ=∑n≥1

δtpre,n, with 0≤tpre,1≤tpre,2 ≤ · · · ≤ tpre,n ≤ · · · ,

where δa is the Dirac measure at a∈R+;(b) A Poisson point process P on R2

+ with rate 1.

See Kingman [31] for general definitions on Poisson processes.The variable t of the point processes Nλ(dt) and P(dx, dt) is interpreted as the

time variable. For t≥0, the σ-field Ft of the filtration (Ft) of the probability spaceis assumed to contain all events before time t for both point processes, i.e.

(4) σ⟨P1

(A×(s, t]

),P2

(A×(s, t]

), A∈B (R+) , s≤t

⟩⊂ Ft.

A stochastic process (U(t)) is adapted if, for all t≥0, U(t) is Ft-measurable. Itis a cadlag process if, almost surely, it is right continuous and has a left limit at


every point t>0, U(t−) denotes the left limit of (U(t)) at t. The Skorokhod spaceof cadlag functions from [0, T ] to S is D([0, T ],S). See Billingsley [5] or Ethier andKurtz [12].

The set of real continuous bounded functions on the metric space S⊂Rd is de-noted by Cb(S), and Ckb (S)⊂Cb(S) is the set of bounded, k-differentiable functionson S with respect to each coordinate, with the respective derivatives are boundedand continuous.

We conclude this preliminary section with an elementary but important lemmaconcerning the filtering of a stochastic process with an exponential function.

Lemma 1 (Exponential Filtering). If µ is a non-negative Radon measure on R+,α>0 and x0∈R, then

H(t) = h0e−αt+

∫(0,t]

e−α(t−s)µ(ds)

is the unique cadlag solution of the differential equation

dH(t) = −αH(t) dt+ µ(dt),

such that H(0)=h0, i.e. satisfying the relation

H(t) = h0−α∫ t

0

H(s) ds+ µ((0, t]), ∀t≥0.

This type of process is a central object in mathematical models of neuroscience.It is used to represent leaky-integrate phenomena of chemical components withincells. See Lapicque [34] and for a general review Gerstner et al. [19]. The process(H(t)) is also referred to as the shot-noise process associated to the measure µ,see Gilbert and Pollak [20].

2.2. The Dynamics of the Post-synaptic Membrane Potential. It is rep-resented as a cadlag stochastic process (X(t)) following leaky-integrate dynamicsillustrated in Figure 1a:

(a) It decays exponentially to 0 with a fixed exponential decay, set to 1.(b) It is incremented by the current synaptic weight variable at each firing

instant of the input neuron, i.e. at each instant of the Poisson point pro-cess Nλ.

(c) The firing mechanism of the output neuron is driven by a function β from Rto R+, the activation function of the neuron. When the membrane potentialis x, the output neuron fires at rate β(x). This function is usually assumedto be non-decreasing, in other words, the larger the membrane potential is,the more likely the neuron is to spike.

After a post-synaptic spike, the neuronal membrane potential X is de-creased by the amount g(x), where g is some function on R. In general, themembrane potential is reset to 0 after a spike, i.e. g(x)=x, see Robert andTouboul [45]. However, in some cases, the reset potential may not dependon the membrane potential before the spike, g can be constant for example.

Post-synaptic spikes. If the instants of pre-synaptic spikes are represented bythe Poisson processNλ, the firing instants of the output neuron tpost,n are expressed


as the points of the point process Nβ,X on R+ defined by

(5)

∫R+

f(u)Nβ,X(du)def.=

∫R+

f(u)P( (

0, β(X(u−))],du)

=

∫R2

+

f(u)1s∈(0,β(X(u−))]P(ds,du),

for any non-negative Borelian function f on R+.Classical properties of Poisson processes give that, for t>0 and x∈R,

P(Nβ,X(t, t+ dt)6=0

∣∣∣∣X(t−)=x

)=β(x) dt+o(dt),

as expected, Nβ,X is an inhomogeneous Poisson process with intensity functionx 7→β(X(t)). See Kingman [31].

The following stochastic differential equation summarizes the description of thetime evolution of (X(t)) given by a), b) and c) in terms of a stochastic differentialequation (SDE),

dX(t) = −X(t) dt+W (t−)Nλ(dt)− g(X(t−))Nβ,X(dt).

It can be written under an integral form as

(6) f(X(t)) = f(X(0))−∫ t

0

f ′(X(s))X(s) ds

+

∫(0,t]

[f(X(s−)+W (s−)

)−f(X(s−)

)]Nλ(ds)

+

∫(0,t]

[f(X(s−)−g(X(s−))

)−f(X(s−)

)]P( (

0, β(X(s−))],ds),

for any C1b (R)-function f .

2.3. Time Evolution of the Synaptic Weight. In this work, the synaptic weightW will be chosen to stay in a real (not necessarily bounded) interval KW . Forseveral examples, the plasticity process leads to dynamics for which the process(W (t)) stays in KW for all time t.

— Taking KW=R leads to free dynamics of the synaptic weight, that canbe either negative or positive, changes it signs because of the plasticityrules. This situation occurs in models of neural networks where excita-tory/inhibitory neurons are not separated in distinct classes..

— If KW=R+, the synaptic weight is non-negative and plasticity processescannot change its sign. This is a model for excitatory neurons whose spikeslead to the increase of the post-synaptic membrane potential.

— Conversely, if KW=R−, the cell is an inhibitory neuron, which has theopposite effect on the post-synaptic membrane potential.

— Finally, KW can also be bounded in order to represent saturation mecha-nisms, i.e. the synaptic weights needs to stay in a biological range of value.

We can now introduce the notion of plasticity kernels.


Definition 1 (Plasticity Kernel). A plasticity kernel Γ is a measurable functional

Γ: Mp(R+)2 −→M+(R+)

(m1,m2) −→ Γ(m1,m2),

M+(R+) is the set of positive Radon measures on R+ and, for any t>0, the func-tional

(7) (m1,m2) −→ Γ(m1,m2)(du ∩ [0, t])

is Gt⊗Gt-measurable, where µ(du∩[0, t]) denotes the restriction of the Radon mea-sure µ to the interval [0, t] and (Gt) is the filtration on Mp(R+), such that for t≥0,Gt is the σ-field generated by the functionals m→m((0, s]), with s≤t.

If Γ is a plasticity kernel andm1, m2∈Mp(R+), the measure Γ(m1,m2)(du∩[0, t])depends only on the variables mi([0, s]), for i∈1, 2 and s≤t.

In our model, the infinitesimal elements at time t for the update of plasticity areexpressed as Γ(Nλ,Nβ,X)(dt) for some plasticity kernel Γ. This quantifies how theinteraction between the instants of pre-synaptic and of post-synaptic spikes, Nλand Nβ,X leads to specific synaptic changes. For example, the order and timingbetween instants of pre- and post-synaptic spikes may have an impact on the typeof plasticity that is elicited.

Plasticity is represented as a process, integrating, with some decay, the past in-teractions of the spiking activity on either side of the synapse. Two non-negativeprocess are introduced: (Ωp(t)) and (Ωd(t)), the first one is associated to potentia-tion (increase of W ) and the other to depression (decrease of W ). They are definedas follows: For a∈p, d,

(8) Ωa(t) = Ωa(0)e−αt +

∫(0,t]

e−α(t−s)Γa(Nλ,Nβ,X)(ds),

where α>0 and the variables Γp and Γd are plasticity kernels associated to potenti-ation and depression respectively. The process (Ωa(t)) can be seen as a exponentialfiltering of the random measure Γa(Nλ,Nβ,X)(dt) in the sense of Lemma 1, i.e. asthe solution of the differential equation

dΩa(t) = −αΩa(t) dt+ Γa(Nλ,Nβ,X)(dt).

In Section A.1 of Appendix, another stochastic model of plasticity with no expo-nential filtering of the plasticity kernels is introduced and discussed.

The time evolution of (W (t)) depends then on the past activity of the inputand output neurons, through (Ωp(t)) and (Ωd(t)) and is described by the followingequation,

(9)dW (t)

dt= M (Ωp(t),Ωd(t),W (t))

As explained in the introduction of this section, the function M need to be chosenso that the synaptic weight W stays at all time in its definition interval KW .

This can be formalized by the following hypothesis. For any piecewise-continuouscadlag functions ωp(t) and ωd(t)) on R+, a solution (w(t)) of the ODE

dw(t)

dt= M(ωp(t), ωd(t), w(t)),

with w(0)∈KW , is such that w(t)∈KW , for all t≥0.


We now give some examples of functions M associated to different synapticdomains KW . For KW=R, we can chose the additive implementation of STDPrules, where,

(10) M(ωp, ωd, w)def.= M(ωp, ωd) = ωp − ωd

In that case, the dynamics are unbounded and we see the update only depends onthe potentiation/depression plasticity variables Ωa.

If we want to model bounded synaptic weight in KW=[Ad, Ap], we can considerthe function M given by

(11) M(ωp, ωd, w)def.= (Ap−w)nωp−(w−Ad)nωd − δ(w −Ar), w∈[Ad, Ap],

where Ad≤Ar≤Ap, and n≥0. This corresponds to a multiplicative influence of W .See Gutig et al. [24]. It is straightforward to see that in that case, the synapticweight stays bounded between Ap and Ad for any plasticity processes Ωa. Theexpression −δ(W (t) − Ar) is for the exponential decay of the synaptic weight Wto Ar, its resting value. This term represents homeostatic mechanisms, i.e. mecha-nisms that maintain steady internal physical and chemical conditions to allow thefunctioning of the system. See Turrigiano and Nelson [56].

Finally, an unbounded dynamics for an excitatory synapse, therefore withKW=R+ can be enforced by,

(12) M(ωp, ωd, w)def.= ωp−wωd,

2.4. Examples of Plasticity Kernels. We show that several important STDPrules of the literature can be represented with plasticity kernels Γp and Γd.

2.4.1. Pair-Based Models. For pair-based mechanisms, the synaptic weight ismodulated according to the respective timing of pre-synaptic and post-synapticspikes, as illustrated in Figure 1b. This is based on the fact that most STDPexperimental studies are based on pairing protocols, where pre- and post-synapticspikes are repeated at a certain frequency for a given number of pairings.

Accordingly, a large class of models have been developed on the principle thatthe synaptic weight change due to a pair (tpre, tpost) of instants of pre- and post-synaptic spikes, depends only on ∆t=tpost−tpre. The synaptic update is then takenproportional to Φ(∆t), where Φ is some non-increasing function converging to 0 atinfinity, that is referred to as the STDP curve. An example of exponential STDPcurves is given in Figure 1b (top left). Many pair-based models have been developedover the years, varying mainly which pairs of spikes are taken into account whenupdating the synaptic weight.

We start with the simplest rule, the all-to-all version (following Morrison et al.[37] terminology), where all pairs of spikes gives an update of the synaptic weight.

All-to-all Model. The all-to-all scheme consists in updating the synaptic weightat each post-synaptic spike, occurring at time t by the sum over all previous pre-synaptic spikes occurring at time s<t of the quantity Φ(t−s). Similarly, by switch-ing the role of pre-synaptic and post-synaptic spikes, the synaptic weight is updatedin the same way with other constants. See Figure 1b (bottom left) for an exampleof all-to-all interactions.


The corresponding plasticity kernels are defined by, for m1, m2∈Mp(R+) anda∈p, d,

(13) ΓPAa (m1,m2)(dt)

def.=

(∫(0,t)

Φa,2(t−s)m2(ds)

)m1(dt)

+

(∫(0,t)

Φa,1(t−s)m1(ds)

)m2(dt).

The functions Φa,i, a∈p, d and i∈1, 2 are non-negative and non-increasing func-tions functions converging to 0 at infinity.

If f is a non-negative Borelian function on R+, we have∫R+

f(t)ΓPAa (m1,m2)(dt)

=∑

u:m1(u)6=0

f(u)∑v:v<u

m2(v) 6=0

Φa,2(u− v)+∑

u:m2(u)6=0

f(u)∑v:v<u

m1(v) 6=0

Φa,1(u− v).

Recall that in our model, the couple (m1,m2) is (Nλ,Nβ,X).

Remarks.

(a) The exponential STDP functions Φ(s)=B exp(−γs), s≥0, are often used inthis context. See Morrison et al. [37]. Several studies also consider the casewhen Φ is a translated exponential kernel. See Lubenov and Siapas [35].

(b) STDP Hebbian rules have been used in many computational models. Forpair-based models, a pre-post pairing represents the causal activation of thecircuit, a pre-synaptic spike, and leads to a spike of the post-synaptic neu-ron. Following Hebb’s postulate, such pairing should lead to potentiationfor a STDP Hebbian rule,

ΓPAHp (m1,m2)(dt) =

(∫(0,t)

Φp,1(t−s)m1(ds)

)m2(dt).

Conversely, a post-pre pairing, a pre-synaptic spike occurs after a post-synaptic one (representing anti-causal activation), leads to depression,

ΓPAHd (m1,m2)(dt) =

(∫(0,t)

Φd,2(t−s)m2(ds)

)m1(dt).

This corresponds to Φp,2=0 and Φd,1=0 in Equation (13). This setting iscommonly referred to as Hebbian STDP, it has been shown experimentally,see Bi and Poo [4].

However, other forms of STDP have been discovered experimentally.For example, anti-Hebbian STDP rules for which Φp,1=0 and Φd,2=0,symmetric LTD rules correspond to Φp,1=Φp,2=0 and symmetric LTP byΦd,1=Φd,2=0. See Feldman [14]. This is the motivation of the generalsetting defined in Equation (13).

(c) Pre/post-synaptic-only plasticity rules can also be expressed into this for-malism. These models include a component to express the direct influenceof the pre- or post-synaptic spikes on the plasticity without any interaction


between the two spike trains. In that case, the kernel ΓPA1a would have the

following expression, for m1, m2∈Mp(R+) and a∈p, d,

(14) ΓPA1a (m1,m2)(dt)

def.=

(∫(0,t)

Φa,2(t−s)m2(ds)

)m1(dt)

+

(∫(0,t)

Φa,1(t−s)m1(ds)

)m2(dt) +Da,1m1(dt) +Da,2m2(dt),

where the constants Da,i, a∈p, d, i∈1, 2, are non-negative.

Nλ, tpre Nβ,X , tpost

Pre-synaptic Post-synaptic

W (t)

−g(x)

(a) A Simple Stochastic Model for a Synaptic Connection

NearestSymmetric

Pre

-synapti

c

Nλ

,t pre

Post

-synapti

c

Nβ,X

,t post

All-to-AllNearestReduced

∆t = tpost − tpre

1/γp,11/γd,11/γd,2

Bp,1

Bd,1

Bd,2

(b) Synaptic Plasticity Kernels for Pair-Based Rules

Figure 1. Stochastic Models of STDP

For the class of models defined by Relation (13) the plasticity at time t is a func-tional of all pairs of pre/post-synaptic and post/pre-synaptic spikes before time t.

A large spectrum of different pair-based models have been used in the literatureranging from the all-to-all model (described above) when all spikes are taken intoaccount, to the nearest models when only the nearest spikes are considered in theplasticity rules. See Morrison et al. [37] for a classification of the different schemes.


Nearest Neighbor Symmetric Model. In the nearest neighbor symmetricmodel, whenever one neuron spikes, the synaptic weight is updated by only takinginto account the last spike of the other neuron, as can be seen in Figure 1b (topright). If the pre-synaptic neuron fires at time tpre, the contribution to the plas-ticity kernel is Φa,2(tpre−tpost) , where tpost is the instant of the last post-synapticspike before tpre.

In our framework, the corresponding kernels ΓPS are defined by, for m1,m2∈Mp(R+) and a∈p, d,

(15) ΓPSa (m1,m2)(dt)

def.= Φa,2(t0(m2, t))m1(dt)+Φa,1(t0(m1, t))m2(dt),

with the following definition, for m∈Mp(R+) and t>0,

(16) t0(m, t) = t− sups : s<t,m(s)6=0,

with the convention that t0(m, 0)=+∞. The quantity t0(m, t) is the delay betweent and the last point of m before t.

Nearest Neighbor Reduced Symmetric Model. For the nearest neighbor sym-metric reduced, a pre-synaptic spike at t is paired with the last post-synaptic spikeat s≤t, only if there are no pre-synaptic spikes in the time interval (s, t). Thisscheme applies in a similar way for post-synaptic spikes. See Figure 1b (bottomright).

In our framework, the kernels ΓPR are defined, for m1, m2∈Mp(R+) anda∈p, d, by

(17) ΓPRa (m1,m2)(dt)

def.=[Φa,2(t0(m2, t))1t0(m2,t)≤t0(m1,t)

]m1(dt)

+[Φa,1(t0(m1, t))1t0(m1,t)≤t0(m2,t)

]m2(dt),

with same notations as in (15). For t>0, the inequality t0(m2, t)<t0(m1, t) is equiv-alent to the relation m1((t0(m2, t), t))=0 so that there is a unique point of m1

paired to t0(m2, t) as expected, and similarly by switching m1 and m2. The up-dates described by Relation (17) are therefore done only for consecutive pre- andpost-synaptic spikes.

Extensions of pair-based models are presented in 2.4.3 and Section 2.4.5

2.4.2. Calcium-Based Models. Pair-based models are phenomenological modelsof STDP in the sense that experimental STDP curves are taken as a core parameterof the models. Another important class of synaptic models are derived from biolog-ical phenomenons and aims at reproducing experimental STDP curves using simplebiological models. A common hypothesis is to use the calcium concentration in thepost-synaptic neuron as a key parameter to model STDP, see Shouval et al. [53]and Graupner and Brunel [21]. Several biophysical models have studied the linkbetween calcium concentration, and its direct implication on the dynamics of theplasticity. A calcium-based model with saturation mechanisms has investigated thedependency on the number of pairings and the existence of different mechanismsfor plasticity in Vignoud et al. [57].

For these models, synaptic plasticity is expressed as a functional of the post-synaptic calcium concentration. For m1, m2∈Mp(R+), the points of m1, resp. thepoints of m2, elicit calcium transfers of amplitudes C1, resp. C2, followed by anexponential decay with rate γ. If (Cm(t)) is the process of the calcium concentration


associated to the couple m=(m1,m2), it is therefore the solution of the differentialequation

dCm(t) = −γCm(t) dt+ C1m1(dt) + C2m2(dt),

with some fixed initial condition. By Lemma 1, it can be expressed as

(18) Cm(t)def.= Cm(0)e−γt+C1

∫(0,t]

e−γ(t−s)m1(ds)+C2

∫(0,t]

e−γ(t−s)m2(ds).

The mechanisms for potentiation, resp. depression, are triggered depending on thecalcium concentration. For a∈p, d, the plasticity kernel ΓC

a is defined by,

(19) ΓCa (m1,m2)(dt)

def.= ha(Cm(t)) dt,

for some non-negative function ha on R+. The function ha is usually a thresholdfunction of the type

(20) ha(x)def.= Ba1x≥θa, x≥0,

for some Ba∈R+ and θa≥0, as done in Graupner and Brunel [21].For the model with threshold functions, the process (Ωa(t)) associated to ΓCa has

therefore an impact on the synaptic weight as soon as the concentration of calciumis above level θa.

2.4.3. Suppression Models. Computational models of pair-based rules of Sec-tion 2.4.1 are easy to implement in large neural networks and they capture someessentials properties of STDP.

Nevertheless, they have been shown to fit poorly with experimental data whenmore complex protocols are used. See Froemke and Dan [17] and Pfister and Gerst-ner [42]. For this reason, more detailed models taking into account the influence ofseveral pre- and post-synaptic spikes have been proposed. Babadi and Abbott [3] isa review of these so-called ‘triplet-based’ rules and their influence on the stabilityof the synaptic weights distribution. The model of this section is a variant of thepair-based model with an additional dependence on earlier instants of post- andpre-synaptic spikes. Another variant is described in Section 2.4.4.

It was observed, using triplet-based protocols in Froemke and Dan [17], thatpreceding pre- and post-synaptic spikes have a ‘suppression’ effect on the HebbianSTDP observed. Motivated by these experiments, the following model, extendingpair-based rules, has been proposed.

If there is a pre-synaptic spike, resp. post-synaptic spike, at time t≥0, we de-note by `1(t) [resp. `2(t)] the instant of the last pre-synaptic, resp. post-synapticspike, before t. For this model, when a pre-synaptic spike occurs at time t≥0, thecontribution to ΓS

a(·, ·)(dt) is the sum over all post-synaptic spikes before time s≤tof the quantities

(1−ΦS,1(t−`1(t))) (1−ΦS,2(s−`2(s))) Φa,2(tpre−s),

and similarly for post-synaptic spikes, where ΦS,i is a non-negative non decreas-ing function verifying ΦS,i(0)≤1 and limt→+∞ ΦS,i(t)=0, for i∈1, 2. In particu-lar, if the instants t1 and t2 of consecutive pre-synaptic spikes are too close, i.e.t2−t1=t2−`1(t) is small, the synaptic weight is not significantly changed at theinstant t2. And similarly for consecutive post-synaptic spikes.


More formally, the plasticity kernels ΓSa, a∈p, d, are defined by, for m1,

m2∈Mp(R+),

ΓSa(m1,m2)(dt)

def.=[

(1−ΦS,1(t0(m1, t))

∫(0,t)

(1−ΦS,2(t0(m2, s))) Φa,2(t−s)m2(ds)

]m1(dt)

+

[(1−ΦS,2(t0(m2, t))

∫(0,t)

(1−ΦS,1(t0(m1, s))) Φa,1(t−s)m1(ds)

]m2(dt)

with the t0(m, t) defined by Equation (16).

2.4.4. Triplet-Based Models. Pfister and Gerstner [42] shows that precedingpre-synaptic spikes enhance the depression obtained for a post-pre pairing, whereaspreceding post-synaptic spikes lead to a bigger potentiation than in a classical pre-post pairing. The plasticity kernels ΓT

a , a∈p, d of the associated model are definedby, for m1, m2∈Mp(R+),

(21) ΓTa (m1,m2)(dt)

def.=(

1+

∫(0,t)

ΦT,a,1(t−s)m1(ds)

)(∫(0,t)

Φa,2(t−s)m2(ds)

)m1(dt)

+

(1+

∫(0,t)

ΦT,a,2(t−s)m2(ds)

)(∫(0,t)

Φa,1(t−s)m1(ds)

)m2(dt).

where, for a∈p, d, i∈1, 2, ΦT,a,i is a non-negative non-decreasing function con-verging to 0 at infinity.

It is interesting to note that this model is in opposition to the suppression modeldescribed just before. Both models are based on experimental data from differentneuronal cells: visual cortical in Froemke and Dan [17], and hippocampal in Pfis-ter and Gerstner [42]. A global model taking into account both mechanisms, theNMDA-model, is defined in Babadi and Abbott [3].

2.4.5. Voltage-Based Models. In Clopath and Gerstner [9], another class ofplasticity rules, voltage-based models, has been used to explain plasticity withbiophysical mechanisms, similarly to calcium-based models.

In particular, filtered traces of the membrane potentialX are used in the synapticupdate. Adapting notations from Clopath and Gerstner [9], we have for depression,

Γd(dt) =

Bd(∫(0,t)

e−γd,2(t−s)X(t− s) ds−θd

)+Nλ(dt),


and for potentiation,

Γp(dt) = Bp

(∫(0,t)

e−γp,0(t−s)X(t−s) ds−θp

)+

×

(∫(0,t)

e−γp,2(t−s)X(t−s) ds−θd

)+

×

(∫(0,t)

e−γp,1(t−s)Nλ(ds)

)dt.

See Relations (1) and (2) of Clopath and Gerstner [9].In their model, an adaptive-exponential integrate-and-fire model (AdEx) is used

to represent the post-synaptic neuron, instead of a Poisson point process. They takeθp above the threshold potential of the AdEx model, leading to a simple estimationin terms of the post-synaptic spike train:

(∫(0,t)

e−γp,0(t−s)X(t− s) ds−θp

)+

dt ∼ Nβ,X(dt).

However, θd lies around the resting potential of the neuron, leading to synapticupdate that are functions of X directly and not only of the spike-trains. Thisfeature justifies the denomination voltage-based models and is not easily taken intoaccount in the framework presented here. To include such a STDP rule, one couldextend the definition of a plasticity kernel to Γ(m1,m2,m) by adding a directdependence on a cadlag adapted process x(t).

We present a variation of the voltage-based model using filtered functionals of pre-and post-synaptic spike trains that fits in our formalism. Notice that both modelsare not equivalent in the sense that in Clopath and Gerstner [9], sub-threshold-activity can lead to plasticity, whereas our model needs post-synaptic spikes.

If there is a pre-synaptic spike at time t>0, the synaptic weight is depressed bythe quantity

Bd

(∫(0,t)

e−γd,2(t−s)Nβ,X(ds)−θd

)+

,

where, for x∈R, x+= max(x, 0), and if some filtered variable is above some thresholdθd at that time.

If there is a pre-synaptic spike at time t, the synaptic weight will be potentiatedby a quantity involving the product of two filtered variables,

Bp

(∫(0,t)

e−γp,2(t−s)Nβ,X(ds)−θd

)+ ∫(0,t)

e−γp,1(t−s)Nλ(ds),


The plasticity kernels are thus defined by, for m1, m2∈Mp(R+),

ΓVd (m1,m2)(dt)

def.=

Bd(∫(0,t)

e−γd,2(t−s)m2(ds)−θd

)+ m1(dt),

ΓVp (m1,m2)(dt)

def.=Bp(∫

(0,t)

e−γp,2(t−s)m2(ds)−θd

)+(∫(0,t)

e−γp,1(t−s)m1(ds)

) m2(dt).

2.5. The Plasticity Process. This section is devoted to the formal definition ofthe stochastic process describing the time evolution of the synaptic weight.

Definition 2. The stochastic process (X(t),Ωp(t),Ωd(t),W (t)) with initial state(x0, ω0,p, ω0,d, w0), is the solution in in D(R+,R×R2

+×KW ) of the SDEs, for t>0,

(22)

dX(t) = −X(t) dt+W (t−)Nλ(dt)− g(X(t−))Nβ,X(dt),

dΩa(t) = −αΩa(t) dt+ Γa(Nλ,Nβ,X)(dt), a∈p, d,dW (t) = M (Ωp(t),Ωd(t),W (t)) dt,

where, Γp and Γd are plasticity kernels and Nβ,X is the point process defined byRelation (5) and the function M is expressed by Relation (9).

The system (22) is a kind of fixed point equation for the process (X(t)) with anintricate dependence due to the point process Nβ,X as an argument of the plasticitykernels. Theorem 1 gives an existence and uniqueness result for the solutions ofEquations (22). We now introduce the main assumptions on the parameters of ourmodel which will be used throughout this paper.

Examples of different dynamics are presented in Section B of Appendix, fordifferent plasticity kernels, pair-based in Figure 3 and calcium-based in Figure 4.

Assumptions A

(a) Firing Rate Function.It is assumed that β is a non-negative, continuous function on R and thatβ(x)=0 for x≤−cβ≤0 and there exists Cβ≥0 such that

(23) β(x)≤Cβ(1+|x|), ∀x∈R.

(b) Drop of Potential after Firing.The function g is continuous on R and 0≤g(x)≤max(cg, x) holds for allx∈R, for some cg≥0.

(c) Dynamic of Plasticity.The function M is such that, for any w∈KW and any cadlag piecewise-continuous functions h1 and h2 on R+, the ODE

(24)dw(t)

dt=M(h1(t), h2(t), w(t)) with w(0)=w,

for all points of continuity of h1 and h2, has a unique continuous solution(S[h1, h2](w, t)) in KW .

Theorem 1. Under Assumptions A, the system (22) has a unique cadlag adaptedsolution with initial state (x0, ω0,p, ωd,0, w0) in R×R2

+×KW .


Proof. The construction is done on the successive intervals between two consecutiveinstants of jump of the system, i.e. of the jumps of the process (X(t)). The non-decreasing sequence (sn) of these instants is defined by induction.

The first jump of (X(t)) occurs at time s1 and is defined as the minimum of thefirst jumps of the processes

(25) (Nλ((0, t])) and

(∫(0,t]

P( (

0, β(x0e−u)] ,du)) .

With Relation (24), for 0≤t<s1, we set X(t)=x0e−t and W (t)=S[Ω1

p,Ω1d](w0, t),

with

Ω1a(t)

def.= ω0,a+

∫(0,t)

e−α(t−s)Γa(0, 0)(ds), a∈p, d,

and W (s1)=W (s1−), where 0 is the null point process.

(a) If s1 is the first point of Nλ, define

f1def.= + and X(s1) = x0e

−s1+W (s1−).

(b) If s1 is the first point of the second point process of Relation (25), set

f1def.= − and X(s1) = x0e

−s1−g(x0e−s1).

The mark f1 indicates the nature of the jump occurring at time s1, i.e. if the spikewas fired by the pre- or a post-synaptic neuron.

The process (X(t),Ω1p(t),Ω

1d(t),W (t)) satisfies the equations (22) on the time

interval [0, s1] and, by Relation (4), s1 is a stopping time with respect to thefiltration (Ft).

Assume by induction that, for n≥0, the variables (sk, fk, 1≤k≤n) and theadapted cadlag process (X(t),W (t), t∈[0, sn]) are defined, and sn is a stoppingtime. For a∈p, d, let

(26) Ωn+1a (t)

def.= ωa+

∫(0,t)

e−α(t−s)Γa

(n∑k=1

δsk1fk=+,

n∑k=1

δsk1fk=−

)(ds).

In Definition 1, the Gt⊗Gt measurability property, gives that for any n≥1 and k<n,the process (Ωja(t)) does not depend on the index j∈k, . . . , n on the time interval[0, sk]. The instant sn+1>sn is defined as the minimum of the first jumps of thetwo point processes,

(27) (Nλ([sn, t]), t>sn),Rndef.=

(∫[sn,t]

P[(

0, β(X(sn)e−(u−sn)

)],du], t>sn

).

The fact that sn is a stopping time and the strong Markov property of the Poissonprocesses Nλ and P give that sn+1 is also a stopping time. For sn≤t<sn+1, set

W (t)=S[Ωn+1p ,Ωn+1

d ](W (sn), t−sn) and X(t)def.= X(sn−)e−(t−sn),

and W (sn+1)=W (sn+1−), and

(a) if sn+1 is a point of Nλ, define fn+1=+, and

X(sn+1)def.= X(sn−)e−(sn+1−sn)+W (sn+1−),

(b) Otherwise, we set fn+1=−, and

X(sn+1)def.= X(sn−)e−(sn+1−sn)−g

(X(sn−)e−(sn+1−sn)

).


We have thus defined by induction a stochastic process (X(t),W (t)) on sequenceof time intervals (sn, sn+1), n≥1. We now prove that the process is defined onthe whole real half-line, i.e. that the sequence (sn) is almost surely converging toinfinity.

Non-Explosive Behavior. Denote by E0 the event where the sequence (sn) isbounded and assume that it has a positive probability. On the event E0, almostsurely, only a finite number of points of the Poisson process Nλ may be points ofthe sequence (sn). Therefore, there exists some N0∈N and a subset E1 of E0 ofpositive probability such that for n≥N0, one has fn=−, i.e. the jumps are due tothe second point process of Relation (27) after time sN0

.On the event E1, for n≥N0 one has

|X(sn−)| < |X(SN0)|,

almost surely, because |X| can only decreases when there are no pre-synaptic spikes.Consequently, the successive first instants of jumps of the processes Rn, n≥N0

cannot stay bounded on the event E1, this is a contradiction. The sequence (sn) istherefore converging to infinity almost surely.

A direct consequence of this result is that, from the very definition of the sequence(sn), for any t>0, there exists n0 such that if n≥n0 then

n∑k=1

δsk1fk=+ ∩ [0, t] = Nλ ∩ [0, t] and

n∑k=1

δsk1fk=− ∩ [0, t] = Nβ,X ∩ [0, t],

recall that µ ∩ [0, t] is the measure µ∈M(R+) restricted to the interval [0, t]. Fora∈p, d, again with the Gt⊗Gt measurability property of plasticity kernels, thequantity

Ωna(t)=Ωa(0)+

∫(0,t)

e−α(t−s)Γa (Nλ,Nβ,X) (ds)

is constant for n≥n0, it is defined as Ωa(t). Furthermore, for s≤t and n≥n0,

dW (s) = M(Ωnp (t),Ωnd (s),W (s)

)ds = M (Ωp(t),Ωd(s),W (s)) ds.

We have thus the existence of a solution to Relation (22). The uniqueness is clearon any time interval [0, sn], n≥1, and therefore almost surely on R+.

3. Markovian Plasticity Kernels

In this section we introduce an important subclass (M) of plasticity kernels thatleads to a Markovian formulation of the whole plasticity process. For this class, itturns out that the associated synaptic weight process (W (t)) can be investigatedwith a scaling approach which is often used, sometimes implicitly, in the literatureof physics in neuroscience. As it will be seen, plasticity kernels of pair-based modelsof Section 2.4.1 and of calcium-based models of Section 2.4.2 belong to this subclass.

Definition 3 (Kernels of Class (M)). A plasticity kernel Γ is of class (M) if, form1, m2∈Mp(R+),

(28) Γ(m1,m2)(dt) = n0(z(t)) dt+ n1(z(t−))m1(dt) + n2(z(t−))m2(dt),

where

(a) For i=0, 1, 2, ni is a non-negative measurable function on R`+, where `≥1is a fixed integer;


(b) (z(t)) is a cadlag function with values in R`+, solution of the differentialequation

(29) dz(t) = [−γ z(t)+k0] dt+ k1(z(t−))m1(dt) + k2(z(t−))m2(dt),

— γ∈R`+, ab=(ai×bi) if a=(ai) and b=(bi) in R`+;

— k0∈R`+ is a constant and k1 and k2 are measurable functions from R`+to R`. Furthermore, the (ki) are such that the function (z(t)) has valuesin R`+ whenever z(0)∈R`+.

It is important to note that the function (z(t)) is a functional of the pair (m1,m2).The fact that z(t) stay non-negative is an important feature of class (M) kernels.For example, we may have functions k1 or k2 of the form,

ki(z) = Bi−bizwhere Bi∈R`+, and bi∈0, 1`.

If Γ is of class (M) and (z(t)) is its associated cadlag process, with Relation (29)it is easily seen that, for any t>0, the functional

(Mp(R+)2,Gt⊗Gt) −→ (M+([0, t]),B(M+([0, t])))

(m1,m2) −→Γ(m1,m2)(du ∩ [0, t])

is indeed Gt-measurable, where (Gt) is the filtration of Definition (1).

Proposition 1 (A Markovian Formulation of Plasticity). Under Assumptions A, ifΓa, a∈p, d, are plasticity kernels of class (M) associated to (na,i, ki), i∈0, 1, 2,a∈p, d and γ∈R`+. The solution of Relations (22) of Theorem 1 is such that the

stochastic process (U(t))def.= (X(t), Z(t),Ωp(t),Ωd(t),W (t)) is a Markov process on

SM(`)def.= R×R`+×R2

+×KW , solution of the SDE,

(30)

dX(t) = −X(t) dt+W (t)Nλ(dt)− g (X(t−))Nβ,X (dt) ,

dZ(t) =[−γ Z(t) + k0] dt

+k1(Z(t−))Nλ(dt) + k2(Z(t−))Nβ,X(dt),

dΩa(t) = −αΩa(t)(t) dt+na,0(Z(t)) dt

+na,1(Z(t−))Nλ(dt)+na,2(Z(t−))Nβ,X(dt), a∈p, d,dW (t) = M (Ωp(t),Ωd(t),W (t)) dt.

Proof. Theorem 1 shows the existence and uniqueness of such a process (U(t)). Theprocess (U(t)) is a piecewise deterministic Markov process in the sense of Davis[10] and consequently has the Markov property. See Chapter 2 of Davis [10]. Anexpression of its infinitesimal generator is given in Section C of Appendix.

It should be noted that, due to the dimension of the state space, the Markovproperty of (U(t)) cannot be really used in practice in our analysis. The repre-sentation in terms of SDEs in Relation (30) is in fact more useful since it allows ascaling analysis in terms of an averaging principle.

Motivation for Markovian Kernels. The processes (Ωp(t)) and (Ωd(t)) deter-mining the synaptic plasticity depend on the process (Z(t)) in a non-linear way.The coordinates of (Z(t))=(Zi(t)) may be interpreted as a representation of theconcentration of chemical components created/suppressed by pre-synaptic and/orpost-synaptic spikes, with some leaking mechanism. Calcium is such an example,


see Relation (18). A simple case is when each coordinate of (Z(t)) is associatedeither to pre- or post-synaptic spikes, i.e. it satisfies

dZi(t) = −γiZi(t) dt+BiNλ(dt) or dZi(t) = −γiZi(t) dt+BiNβ,X(dt).

Moreover, if Zi needs to be reset to Bi when one of the neurons spikes, we justneed to replace Bi by Bi−Zi(t−) in these equations.

We now show that calcium-based models and pair-based models, can be repre-sented in such a setting, i.e. that their plasticity kernels are of class (M).

3.1. Calcium-Based Models. For this set of models, the class (M) property isfairly clear. Relations (18) and (19) give that, for a∈p, d and m1, m2∈Mp(R+),

ΓCa (m1,m2)(dt)def.= ha(Cm(t)) dt,

where, if m=(m1,m2), (Cm(t)) is a cadlag solution of the differential equation

dCm(t) = −γCm(t) dt+ C1m1(dt) + C2m2(dt).

The process (Z(t)) is simply the one-dimensional process (CNλ,Nβ,X (t)). Markoviandynamics of the calcium-based model are illustrated in Figure 4-(a).

3.2. Pair-Based Models. Several kernels associated to pair-based models definedby Relation (13) turn out to be also of class (M). This type of Markov propertyhas been mentioned in Morrison et al. [37] for computational models of severalpair-based STDP rules. A list of other Markovian expression for models of STDPdescribed in 2.4 that are of class (M) can be found in Appendix C, along with somemore general Markovian models.

3.3. All-to-all Model. The class (M) holds when the STDP functions Φ areexponential, i.e. when, for a∈p, d and i∈1, 2,

Φa,i(t)=Ba,i exp(−γa,it), t≥0.

with Ba,i∈R+ and γa,i>0. For m1 and m2∈Mp(R+), denote by (za,i(t)), the cadlagsolution of the differential equation

dza,i(t) = −γa,iza,i(t) dt+Ba,imi(dt),

with za,i(0)=0. Lemma 1 gives the relation

za,i(t) = Ba,i

∫(0,t]

e−γa,i(t−s)mi(ds).

The process (z(t)) is then defined as (zp,1(t), zp,2(t), zd,1(t), zd,2(t)). The plasticitykernel of this model, see Relation (13), can be expressed as

ΓPAa (m1,m2) = na,1(z(t−))m1(dt)+na,2(z(t−))m2(dt),

the functions (na,i) are defined by, for z=(za,i)∈R4+, na,1(z)=za,2 and na,2(z)=za,1.

An example of dynamics with plasticity kernels and associated Markov process(Za,i) is presented in Figure 3-(a). Similar models, using auxiliary processes (Za,i)can be devised for nearest STDP rules. See Appendix C, Figure 3-(b) for the nearestneighbor symmetric STDP and Figure 3-(c) for the nearest neighbor symmetricreduced STDP.


3.4. Nearest Neighbor Models. For m∈Mp(R+) and t>0, the variable t0(m, t)of Relation (16) used in the two models presented in Section 2.4.1,

t0(m, t) = t− sups : s<t,m(s)6=0,can be expressed as the solution (zm(t)) of the differential equation,

dzm(t) = dt−zm(t−)m(dt),

with zm(0)=0.For m1 and m2∈Mp(R+), we define (z(t))=(zm1(t), zm2(t)), Relation (29) holds

with γ=(0, 0), k0=(1, 1) and, for z=(z1, z2), k1(z)=(−z1, 0) and k2(z)=(0,−z2).In this setting, both nearest models are of class M:

— The nearest neighbor symmetric model, ΓPSa of Relation (15), with, for

z=(z1, z2), na,0(z)=0, na,1(z)=Φa,2(z2) and na,2(z)=Φa,1(z1).— The nearest neighbor symmetric reduced model, ΓPR

a of Relation (17), withna,0(z)=0, na,1(z)=Φa,2(z2)1z2≤z1 and na,2(z)=Φa,1(z1)1z1≤z2.

4. A Scaling Approach

4.1. Scaling Assumption. As mentioned in the introduction, different timescalesdrive synaptic plasticity and we will use this property to derive a simple tractableversion of the system (30).

On the one hand, the inverse of the rate of decay of membrane potential andthe mean duration between two pre-synaptic spikes or two post-synaptic spikes areof the order of several milliseconds. See Gerstner and Kistler [18]. For plasticitykernels of class (M), the stochastic process (Z(t)) represents, in some way, cellularprocesses, and therefore the associated timescale is of the same order. For exam-ple, pair-based models have an exponential decay whose inverse is of the order of50 milliseconds. See Bi and Poo [4] and Fino et al. [15]. Similarly, for calcium-based models, calcium concentration decays with a time constant of the order of20 milliseconds. See Graupner and Brunel [21].

On the other hand, he synaptic weight (W (t)) changes on a much slowertimescale, It can take seconds and even minutes to observe an effect of a protocolon the synaptic weight. See Bi and Poo [4]. Consequently, the synaptic variablesΩa, a∈p, d, of Equation (8) also vary on such a slow timescale. They integrate,with an exponential decay of the order of seconds or even minutes, the associatedplasticity kernels.

This is a common assumption of many mathematical models of synaptic plas-ticity in the literature in physics. Kempter et al. [30] “have introduced a smallparameter . . . , with the idea in mind that the learning process is performed on amuch slower timescale than the neuronal dynamics.”. In Roberts [47], a separationof the timescales is also assumed:“This separation is appropriate because the mea-surable changes in behavior occur during the course of several training cycles (t),whereas the neuronal activity modulation that is responsible for synaptic change isgreatest within each cycle (x).”. With different neuronal dynamics, but in the sameframework, Kistler and Hemmen [32], Rossum et al. [49], and Rubin et al. [50] alsouse scaling parameters to stress differences of timescales.

4.2. A Scaled Model of Plasticity of Kernels of Class M. To take into ac-count these multiple timescales, a scaling parameter ε>0 is introduced for plasticitykernels of class (M) in the following way.


(a) The exponential decay of (X(t)), (Z(t)) and the rates λ and β(·) are scaledwith the factor 1/ε.

(b) The functions na,i, a∈p, d, i∈1, 2 associated to synaptic updates due toneuron spikes are scaled by ε.

The initial condition of (Uε(t)) is assumed to satisfy the relation

(31) Uε(0) = U0 = (x0, z0, ω0,p, ω0,d, w0).

This leads to the definition of a scaled version of system (30). We denote(Uε(t))=(Xε(t), Zε(t),Ωε,p(t),Ωε,d(t),Wε(t)), a solution of the system:

(32)

dXε(t) = −1

εXε(t) dt+Wε(t)Nλ/ε(dt)−g (Xε(t−))Nβ/ε,Xε (dt) ,

dZε(t) =1

ε

[−γ Zε(t)+k0

]dt

+k1(Zε(t−))Nλ/ε(dt)+k2(Zε(t−))Nβ/ε,Xε(dt),

dΩε,a(t) = −αΩε,a(t) dt+na,0(Zε(t)) dt

+ε

[na,1(Zε(t−))Nλ/ε(dt)+na,2(Zε(t−))Nβ/ε,Xε(dt)

], a∈p, d,

dWε(t) = M (Ωε,p(t),Ωε,d(t),Wε(t)) dt.

From Relations (32), the dynamic of the processes (Ωε,p(t)), (Ωε,d(t)) and (Wε(t))are slow in the sense that the fluctuations within a bounded time-interval are limitedeither because of the deterministic differential element dt with a locally boundcoefficient, or via a driving Poisson process with rate of order 1/ε but with jumpsof size proportional to ε. The processes (Xε(t)) and (Zε(t)) are fast, for each ofthem the fluctuations are driven either by the deterministic differential elementdt/ε, or the jumps of Poisson point processes with rates of the order of 1/ε.

4.3. Averaging Principles. From now on, we will be interested in the limit-ing behavior of the synaptic weight process (Wε(t)) when the scaling parameterε goes to 0. An intuitive, rough, picture of the type of results that can be ob-tained is as follows: for ε small enough, on a small time interval the slow process(Ωp,ε(t),Ωd,ε(t),Wε(t)) is almost constant, and, due to its fast timescale, the pro-cess (Xε(t), Zε(t)) is “almost” at its equilibrium distribution Πw associated to thecurrent value of synaptic weight of the slow process. If this statement holds in anappropriate way, we can then write deterministic ODE for the time evolution of apossible limit of (Ωp,ε(t),Ωd,ε(t),Wε(t)).

These results are routinely used in mathematical models of synaptic plasticityof the literature in physics, for example Kempter et al. [30]:

“According to the strong law of large numbers [Lamperti, 1996] inconjunction with η [ε] being ‘small’ [Sanders and Verhulst, 1985],we can average the resulting equation, viz., Eq. (1), regardless ofthe random process. In other words, the learning procedure is self-averaging. Instead of averaging over several trials, we may alsoconsider one single long trial during which input and output char-acteristics remain constant. Again, if η [ε] is sufficiently small,timescales are separated and learning is self-averaging.”


In a mathematical context, these types of results are referred to as averaging prin-ciples. See Papanicolalou et al. [39] and Chapter 7 of Freidlin and Wentzell [16] forgeneral presentation. See also Kurtz [33] in the context of jump processes. Thereare few rigorous results of this kind for the moment, Helson [26] is one of the ex-ceptions where an averaging principle is proved for a specific STDP rule. We nowintroduce the framework of our main theorem concerning averaging principles.

If we set the process (Wε(t)) to be a constant equal to w, the time evolution of(Xε(t), Zε(t)) in Relation (32) has the Markov property. The corresponding processwill be referred to as the fast process. Its infinitesimal generator is defined by, iff∈C1

b

(R+×R`

), w∈KW and (x, z)∈R×R`+,

(33) BFw (f)(x, z)def.= −x∂f

∂x+

⟨−γ z+k0,

∂f

∂z(x, z)

⟩+ λ

[f(x+w, z+k1(z))−f(x, z)

]+ β (x)

[f(x−g(x), z+k2(z))−f(x, z)

].

We now introduce a set of general assumptions for plasticity kernels of class (M)driving the system (30).

Assumptions B

(a) Γp and Γd are kernels of class M, with,— All coordinates of the vector γ are positive;— There exists Ck≥0 such that 0≤k0≤Ck and functions ki, i=1, 2, inC1b (R`+), are upper-bounded by Ck≥0;

— It is assumed that, there exists a constant Cn such that, for j∈0, 1, 2,a∈p, d, the function na,j is assumed to be non-negative and Boreliansuch that

(34) na,j(z)≤Cn(1+‖z‖),

where, for z∈R`+, ‖z‖=z1+ · · ·+z`.For any w∈KW , the discontinuity points of

(x, z)7→(na,0(z), na,1(z), β(x)na,2(z))

for a∈p, d, are negligible for the probability distribution Πw of Re-lation (33).

(b) M can be decomposed as, M(ωp, ωd, w)=Mp(ωp, w)−Md(ωd, w)−δw, whereMa(ωa, w) is non-negative continuous function, non-decreasing on the firstcoordinate for a fixed w∈Kw, and,

Ma(ωa, w) ≤ CM (1 + ωa),

for all w∈KW , for a∈p, d.Note that, in the list of different STDP models presented in Section C, only

triplet-based models may not verify this property. We can now state the mainresult concerning the scaled model. Its proof is the main result of the companionpaper Robert and Vignoud [46].

Theorem 2 (Averaging Principle). Under Assumptions A and B and for initialconditions satisfying Relation (31), there exists S0∈(0,+∞], such that the familyof processes (Ωp,ε(t),Ωd,ε(t),Wε(t), t<S0), ε∈(0, 1), of the system (32), is tight forthe convergence in distribution.


As ε goes to 0, any limiting point (ωp(t), ωa(t), w(t), t<S0), satisfies the ODEs,for a∈p, d,

(35)

dωa(t)

dt=−αωa(t)+

∫R×R`+

[na,0(z)+λna,1(z)+β(x)na,2(z)

]Πw(t)(dx, dz),

dw(t)

dt=M(ωp(t), ωd(t), w(t)),

where, for w∈KW , Πw is the unique invariant distribution Πw on R×R`+ of the

Markovian operator BFw .If KW is bounded, then S0=+∞ almost-surely.

If Relation (35) has a unique solution for a given initial state, the convergence indistribution of (Wε(t)) when ε goes to 0 is therefore obtained. Such a uniquenessresult holds if the integrand, with respect to s, of the right-hand side of Rela-tion (35) is locally Lipschitz as a function of w(s). One has therefore to investigateregularity properties of the invariant distribution Πw as a function of w. This isa quite technical topic, however methods based on classical results of perturbationtheory, see Chapter 8 of Kato [29] and their generalizations in a stochastic context,see Has’minskiı [25], can be used to prove this type of properties. Several exampleswhere this property holds are investigated in Sections 5 and 6.

The convergence properties are stated on a fixed time interval [0, S0). The reasonis that, for some of our models the variable S0 is finite. The limit in distributionof (Wε(t)), as ε goes to 0, blows-up, i.e. hits infinity in finite time. An ana-logue property holds for some mathematical models of large populations of neuralcells with fixed synaptic strengths. In this case, the blow-up phenomenon is theresult of mutually exciting dynamics of populations of neural cells. In our case,the strengthening of the connection may grow without bounds when the functionz 7→n2(z) exhibits some linear growth with respect to some of the coordinates ofz and when the activation function β has also a linear growth. See Robert andVignoud [46].

The next sections investigate several important examples of pair-based andcalcium-based models in the light of Theorem 2. In order to have simpler expres-sions, we restrict our study in the following sections to the linear neuron withoutreset receiving excitatory inputs, leading to the following set of assumptions,

Assumptions L (Linear)

(a) The initial conditions of Relation (31) are such that U0=(0, 0`, 0, 0, 0);(b) The output neuron is without reset, i.e. the function g is null. The SDE

associated to the membrane potential is

(36) dXw(t) = −Xw(t) dt+ wNλ(dt);

(c) It receives excitatory inputs, i.e. 0⊂KW⊂R+ and M is LM -Lipschitz;(d) The activation function is linear, β(x)=ν+βx, x≥0 for ν≥0 and β>0.

5. Pair-Based Rules

We investigate scaled models of pair-based rules introduced in Sections 2.4.1and 3.2 with Assumptions L. In this setting, explicit expressions of the stationarybehavior of fast processes can be obtained. As a consequence, we will be able toderive a closed form expression of the asymptotic equation (35).


Proposition 2. If (Xw∞(t),−∞≤t≤+∞) is a stationary version of SDE (36), then

the point process Nβ,Xw∞ of Relation (5) extended on the real line is stationary and,if f is a bounded Borelian function with compact support on R,

(37) − lnE[exp

(−∫ +∞

−∞f(s)Nβ,Xw∞(ds)

)]= ν

∫ +∞

−∞

(1−e−f(s)

)ds+λ

∫ +∞

−∞

(1− exp

(−βw

∫ +∞

0

(1−e−f(t+s)

)e−s ds

))ds.

Proof. Set

(38) (Xw∞(t))

def.=

(w

∫ t

−∞e−(t−s)Nλ(ds)

),

it is easily seen that this process is almost surely defined and that it satisfies Rela-tion (36). The stationarity property of (Xw

∞(t)) and, consequently of Nβ,Xw∞ , comesfrom the invariance by translation of the distribution of Nλ.

The independence of P and Nλ, and the formula for the Laplace transform ofPoisson point processes, see Proposition 1.5 of Robert [44] give the relation

E[exp

(−∫ +∞

−∞f(s)Nβ,Xw∞(ds)

)]= E

[exp

(−∫ +∞

−∞

(1−e−f(s)

)β(Xw

∞(s)) ds

)].

If F is a non-negative bounded Borelian function with compact support on R, withRelation (38) and Fubini’s Theorem, we get

(39)

∫ +∞

−∞F (s)Xw

∞(s) ds =

∫ +∞

−∞

(w

∫ +∞

0

F (u+s)e−s ds

)Nλ(du).

We conclude the proof by using again the formula for the Laplace transform of Nλ.

5.1. All-to-all Model. We recall the Markovian formulation of the all-to-all pair-based model with exponential functions Φ. See Section 3.2.

Assumptions PA The process (Z(t)) is in R4+ and na,0≡0, na,1(z)=za,2 and

na,2(z)=za,1, for z∈R4+, for a∈p, d. For w≥0, the fast process associated to

the operator BFw of Relation (33), can therefore be expressed as (Xw(t), Zw(t)),where (Xw(t)) is the solution of Relation (36) and

(40)

dZwa,1(t) = −γa,1Zwa,1(t) dt+Ba,1Nλ(dt),

dZwa,2(t) = −γa,2Zwa,2(t) dt+Ba,2Nβ,Xw(dt),

for a∈p, d, where γ=(γa,i) > 0 and B=(Ba,i) in R4+.

Proposition 3. If, for w≥0, ΠPAw is the invariant distribution of (Xw(t), Zw(t)),

then, for a∈p, d,∫R×R4

+

(na,0(z)+λna,1(z)+β(x)na,2(z))ΠPAw (dx,dz) =

ν

βλΛ1a+(Λ1a+Λ2

a

)w.

with

(41) Λ1a = βλ2

(Ba,1γa,1

+Ba,2γa,2

)and Λ2

a=βλBa1

1+γa,1.


Proof. The existence of ΠPAw is a simple consequence of the fact that the coordinates

of γ are positive and of Proposition 4 of Section 5 of Robert and Vignoud [46].Assume that the initial point of Relation (40) is a random variable (Xw, Zw) withdistribution ΠPA

w .For a∈p, d, it is easily seen that E

[Zwa,1

]=λBa,1/γa,1 and E [Xw] =λw. Denote

(Y w(t))=(Xw(t)Zwa,1(t)), then with Relation (40), we get

dY w(t) = −(1+γa,1)Y w(t) dt+(wZwa,1(t−)+Ba,1X

w(t−)+wBa,1

)Nλ(dt),

by integrating this ODE on [0, t] and taking the expected value, we obtain

(1+γa,1)E[XwZwa,1

]= λwE

[Zwa,1

]+ λBa,1E [Xw] + λwBa,1.

A similar argument for (Zwa,2(t)), gives the relation

γa,2E[Zwa,2

]= Ba,2E [β(Xw)] = Ba,2(ν+λβw).

The proposition is proved.

Theorem 3. If Assumptions L and PA hold then, as ε goes to 0, the process(Ωε,p(t),Ωε,d(t),Wε(t)) of Relation (32) converges in distribution to the unique so-lution (ωp(t), ωd(t), w(t)) of the relations,

ωa(t) =ν

λβΛ1a

1−e−αt

α+(Λ1a+Λ2

a

)e−αt

∫ t

0

eαsw(s) ds, a∈p, d,dw(t)

dt= M (ωp(t), ωd(t), w(t)) ,

where Λia, i∈1, 2, a∈p, d are defined by Relation (41).

Proof. This is a direct consequence of Theorem 2 and of Proposition 3

Note that, for a∈p, d, the parameter Λ1a is proportional to the area under

the two STDP curves Φa,i(x)=Ba,i exp(−γa,i(x)), i=1, 2. It represents the aver-aged potentiation/depression rate as if we had considered two neurons without anyinteractions. Two important facts results from this property,

— the term in the dynamics for the constant firing rate of the output neuron,ν, is proportional to Λ1

a, as expected;— the term Λ2

a reflects the dependence between pre- and post-synaptic spikes.

5.2. Nearest Neighbor Symmetric Model. Similar results can be obtained forthe nearest neighbor symmetric scheme of Section 1b with general STDP curves Φ.

Assumptions PNS The process (Z(t)) is in R2+ and na,0(z)=0, na,1(z)=Φa,2(z2),

na,2(z)=Φa,1(z1), for z∈R2+, for a∈p, d. The functions Φa,1 and Φa,2 are non-

negative, non-increasing and differentiable, for a∈p, d.For w≥0, the fast process associated to the operator BFw of Relation (33), can

be expressed as (Xw(t), Zw(t)), where (Zw(t)) is the solution of the SDEs,

(42)

dZw1 (t) = dt−Zw1 (t−)Nλ(dt),

dZw2 (t) = dt−Zw2 (t−)Nβ,Xw(dt).


Proposition 4. For w≥0, the Markov process (Xw(t), Zw(t)) has a unique invari-ant distribution ΠPS

w then, if f is a bounded Borelian function on R2+ and a≥0,∫

R×R2+

f(x, z1)ΠPSw (dx, dz) = E

[f(we−Eλ(1+S), Eλ

)],∫

R×R2+

1z2≥aΠPSw (dx, dz)= exp

(−νa

(1−e−ξ

)−λ∫ a

0

(1− exp

(−βw

(1−es−a

)))ds

−λ∫ 0

−∞

(1− exp

(−βw

(1−e−a

)es))

ds

),

where Eλ and S are independent random variables, Eλ has an exponential distri-bution with rate λ and, for ξ≥0,

E[e−ξS

]= exp

(−ξλ

∫ +∞

0

ue−ue−ξe−u

du

).

Proof. The first condition of Assumption B-(a) is clearly not satisfied, the coordi-nates of the vector γ being −1. This is not a concern since this condition is onlyused to construct a Lyapounov function as in the proof of Proposition 4 of Section 5of [46]. We only show that one can construct such a function for this model. Set,for (x, z)∈R×R2

+,

H(x, z)def.=

1

xδ+ x+z1+z2,

for some δ∈(0, λ/2), then,

BwF (H) ≤ 1

xδ

(δ+λ

(xδ

(x+w)δ−1

))+λw+2−x−λz1−(ν+β)xz2,

it is not difficult to see that there exists K sufficiently large such that the relationBFw (H)(x, z)≤−1 holds whenever (x, z)6∈[1/K,K]×[0,K]×[0,K2]. In particular His a Lyapounov function for BFw . Consequently, there exists a unique invariantdistribution.

We denote by (Xw, Zw1 , Zw2 ) a random variable with distribution ΠPS

w . With thenotation of Relation (16), for t>0,

(Xw(t), Zw1 (t)) =

(w

∫ t

0

e−(t−s)Nλ(ds), t0(Nλ, t))

dist.=

(w

∫ 0

−tesNλ(ds), t0(Nλ, 0)

),

by letting t go to infinity, we thus get, with t0def.= t0(Nλ, 0),

(Xw, Zw1 )dist.=

(w

∫ 0

−∞esNλ(ds), t0(Nλ, 0)

)=

(we−t0

(1+

∫(−∞,−t0)

es+t0Nλ(ds)

), t0

).

The strong Markov property of Nλ gives the desired relation for the representationof the law of (Xw, Zw1 ). Again, with the formula of the Laplace transform of Poissonpoint processes, we have

E

[exp

(−ξ∫

(−∞,−t0)

es+t0Nλ(ds)

)]=E

[exp

(−ξ∫ 0

−∞esNλ(ds)

)]= exp

(−λ∫ +∞

0

(1−e−ξe

−s)

ds

).


The stationary distribution of (Z2(t)) is the distribution of the distance of the firstpoint of Nβ,X at the left of 0 at equilibrium, hence, for a≥0,

P (Zw2 ≥a) =P(Nβ,Xw∞((−a, 0))=0

)=P(Nβ,Xw∞((0, a))=0

).

Relation (37), gives, for ξ≥0,

− logE[e−ξNβ,Xw∞ ((0,a))

]=νa

(1−e−ξ

)+λ

∫ a

0

(1− exp

(−βw

(1−e−ξ

)(1−es−a

)))ds

+λ

∫ 0

−∞

(1− exp

(−βw

(1−e−ξ

) (1−e−a

)es))

ds.

By letting ξ go to infinity, we obtained the desired expression. The proposition isproved.

Theorem 4 (Averaging Principle). Under Assumptions L and PNS, as ε goes to 0,the process (Wε(t)) of Relation (32) converges in distribution to (w(t)), the uniquesolution of the ODE

dw(t)

dt= M

(∫ t

0

e−α(t−s)∫R×R4

+

(β(x)Φp,2(z1)+λΦp,1(z2)) ΠPSw(s)(dx,dz) ds,

∫ t

0

e−α(t−s)∫R×R4

+

(β(x)Φd,2(z1)+λΦd,1(z2)) ΠPSw(s)(dx,dz) ds, w(t)

),

where ΠPSw is defined in Proposition 4.

Proof. For w≥0, let (Xw∞, Z

w∞,1, Z

w∞,2) be random variables with distribution Πw,

and, for a∈p, d,

Ψa(w)def.= E

[β(Xw

∞)Φa,2(Zw∞,1)]

+λE[Φa,1(Zw∞,2)

]the ODE can be rewritten as

dw(t)

dt= M

(∫ t

0

e−α(t−s)Ψp(w(s)) ds,

∫ t

0

e−α(t−s)Ψd(w(s)) ds, w(t)

).

With Theorem 2, all we have to prove is that this ODE has a unique solution.This is a simple consequence of the Lipschitz property of Ψa. Indeed, firstly, thedistribution of Zw∞,1 does not depend on w and β(·) is an affine function of Xw

∞given by Relation (38). Finally, the identity

E[Φa,1(Zw∞,2)

]= −

∫ +∞

0

Φa,1(u)P(Zw∞,2≤u) du,

Proposition 4, and simple estimations give that the function Ψa has the Lipschitzproperty. The theorem is proved.

6. Calcium-Based Rules

We investigate scaled models of calcium-based rules introduced in Sections 3.1and 3.1. Our goal here is of showing that the asymptotic equation (35) has a uniquesolution. Some regularity properties of the invariant distribution of the operatorBFw , with respect to the variable w, have to be obtained.

Assumptions C In this case, the vector (Z(t)) in this case is a non-negative one-dimensional process (C(t)). For w∈KW , the fast process associated to the operator


BFw of Relation (33) can be expressed as (Xw(t), Cw(t)), where, as before, (Xw(t))is the solution of Relation (36) and the SDE for (Cw(t)) is

(43) dCw = −γCw(t) dt+ C1Nλ(dt) + C2Nβ,Xw(dt)

where C1 and C2≥0, γ>0.In this case, we also assume na,0(c)=ha(c), na,1(c)=0, na,2(c)=0, for c∈R+. The

functions hp and hd are L-Lipschitz.

Proposition 5. For w∈KW , the Markov process (Xw(t), Cw(t)) has a unique in-variant distribution ΠC

w, and its Laplace transform is given by, for a and b≥0,

− ln

∫R2

+

e−ax−bcΠCw(dx,dc) = ν

∫ 0

−∞

(1−e−bC2e

γu)

du

+λ

∫ 0

−∞

(1− exp

(−aweu−bC1e

γu − βw∫ +∞

0

(1−e−bC2e

γ(u+s))e−s ds

))du.

Proof. The existence and uniqueness of ΠCw is a direct consequence of Theorem 2

since Assumptions B hold in this case and Proposition 4 of Section 5 of [46] can beused.

With Proposition 2 and Lemma 1, the stationary version (Xw∞(t), Cw∞(t)) of the

process (Xw(t), Cw(t)) can be represented as

(44)

(w

∫ t

−∞e−(t−s)Nλ(ds), C1

∫ t

−∞e−γ(t−s)Nλ(ds)+C2

∫ t

−∞e−γ(t−s)Nβ,Xw∞(ds)

).

Hence, we have to calculate E [exp(−aXw∞(0)−bCw∞(0))], that is

Ψ(a, b)def.= E

[exp

(−∫ 0

−∞(awes+bC1e

γs)Nλ(ds)−bC2

∫ 0

−∞eγsNβ,Xw∞(ds)

)].

We proceed as in the proof of Proposition 2, by independence of P and Nλ,

E[exp

(−bC2

∫ 0

−∞eγsNβ,Xw∞(ds)

)∣∣∣∣Nλ]= exp

(−∫ 0

−∞

(1−e−bC2e

γs)β(Xw

∞(s)) ds

)and, with the help of Relation (39), we follow the same methods to obtain thedesired result.

The proposition is proved.

Theorem 5. Under Assumptions L and C, as ε goes to 0, the family of processes(Wε(t)) is converging in distribution to (w(t)), the unique solution of the ODE

(45)dw(t)

dt= M

(∫ t

0

e−α(t−s)∫R2

+

hp(c)ΠCw(s)(dx, dc) ds,

∫ t

0

e−α(t−s)∫R×R+

hd(c)ΠCw(s)(dx,dc) ds, w(t)

),

almost-surely, where, for w∈KW , ΠCw is the probability distribution defined in

Proposition 5.

Proof. The application of Theorem 2 is straightforward. All we have have to provenow is that ODE (45) has a unique solution.


From the representation (44), for any 0≤w≤w′, the two random variables

(Cw∞(0)) and (Cw′

∞ (0)) can be constructed on the same probability space. TheLipschitz property of ha, with the constant L, gives

da(w,w′)def.=∣∣∣E [ha(Cw∞(0))]−E

[ha(Cw

′

∞ (0))]∣∣∣ ≤ LaE [|Cw∞(0)−Cw

′

∞ (0)|]

= C2LE[∣∣∣∣∫ 0

−∞eγsNβ,Xw∞(ds)−

∫ 0

−∞eγsNβ,Xw′∞ (ds)

∣∣∣∣] .with Equation (38), we have Xw

∞(t)=wX1∞(t) for all t and, therefore,

da(w,w′)

C2L≤ E

[∫ 0

−∞eγsP

[(β(wX1

∞(s)), β(w′X1∞(s))

],ds]]

= β(w′−w)E[∫ 0

−∞eγsX1

∞(s) ds

]=β

γ(w′−w)

Let (w(t)), (w′(t)) be two solutions of ODE (45) with the same initial point, then

∆a(t)def.=

∣∣∣∣∣∫ t

0

e−α(t−s)

[∫R×R+

ha(c)ΠCw(s)(dx, dc)−

∫R×R+

ha(c)ΠCw′(s)(dx, dc)

]ds

∣∣∣∣∣≤∫ t

0

∣∣∣E [ha(Cw(s)∞ (0))

]−E

[ha(Cw

′(s)∞ (0))

]∣∣∣ ds ≤ C2Lβ

γ

∫ t

0

|w(s)−w′(s)|ds.

With Relation (45) and the Lipschitz property of M , we get, for t≤T ,

‖w−w′‖tdef.= sup

s≤t|w(s)−w′(s)| ≤ LM

∫ t

0

e−δ(t−s)(∆p(s)+∆d(s)) ds

≤ 2LMC2LTβ

γ

∫ t

0

‖w−w′‖s ds.

This implies that (‖w−w′‖t) is identically 0. The theorem is proved.

The Lipschitz assumptions for the functions (hp, hd) of Assumptions C do notapply to the classical threshold functions (Sθp , Sθd) defined by Relations (20). Ad-ditionally, even in the case of Lipschitz functions, the quantities∫

R×R+

ha(c)ΠCw(dx, dc), a∈p, d,

of the ODE (45) does not have a closed form expression in general in models withoutreset. As we have seen in Section 5.1, for all-to-all pair-based models, only momentsof invariant distributions were required.

For this reason, the next section introduces a class of discrete calcium-basedmodels for which the invariant distributions have an explicit expression which canbe used in practice.

In a biological context, with calcium fluorescence indicators, such as GCaMPfor example Nakai et al. [38], it is now possible to monitor calcium concentrationin dendrites of post-synaptic neurons during stimulations. See Higley and Sabatini[27]. It is thus possible to infer these cumulative functions from such experiments.

From the point of view of numerical analysis, it is quite difficult to obtain somesimple numerical results to express solutions of the ODE (45). It could be done,


by simulations, to estimate the quantities EΠCw

(ha(C)), a∈p, d, for a large num-ber of values for w. A recent article, see Graupner et al. [23], has derived someapproximations for some special cases.

7. Discrete Models of Calcium-Based Rules

In this section, we introduce a simple model of plasticity where the membranepotential X, the calcium concentration C and the synaptic weight W are integer-valued variables. It amounts to represent these three quantities X, C and Was multiple of a “quantum” instead of a continuous variable. This is in fact abiologically plausible assumption for potential and calcium.

The leaking mechanism in particular, the term corresponding to −γZ(t)dt inthe continuous model, Z∈X,C,W and γ>0, in the SDEs, is represented by thefact that each quantum leaves the cell/synapse at rate γ. It should be noted thatsuch a discrete setting can be easily extended to all Markov plasticity models. Wejust consider one calcium variable for the sake of simplicity and also because ouraveraging principle for calcium-based models, Theorem 5, is less satisfactory thanthe analogue result for pair-based models of Section 5.

A comparison between continuous and discrete models of calcium-based STDPis presented in Section B of Appendixand illustrated by Figure 4. The state of thesystem is associated to the solution of the following SDEs

dX(t) = −X(t−)∑i=1

N1,i(dt) +W (t−)Nλ(dt)−X(t−)∑i=1

Nβ,i(dt),

dC(t) = −C(t−)∑i=1

Nγ,i(dt) + C1Nλ(dt) + C2

X(t−)∑i=1

Nβ,i(dt),

dΩa(t) = (−αΩa(t)+ha(C(t))) dt, a∈p, d,

dW (t) = −W (t−)∑i=1

Nδ,i(dt) +ApNΩp(t−)(dt)−Ad1W (t−)≥AdNΩd(t−)(dt),

where C1, C2∈N and, for a∈p, d, Aa∈N and ha is a non-negative function.For ξ>0, Nξ, resp. (Nξ,i), is a Poisson process on R+ with rate ξ, resp. an i.i.d.

sequence of such point processes. For a∈p, d, as before, the notation NΩa(t−)(dt)

stands for P [(0,Ωa(t−)),dt], where P is a Poisson process in R2+ with rate 1. All

Poisson processes are assumed to be independent.We have taken g(·) as the constant function equal to 1. As it can be seen,

the firing rate in the evolution of (X(t)) is the linear function x 7→βx. The timeevolution of the discrete random variable (W (t)) is driven by two inhomogeneousPoisson processes, one for potentiation and the other for depression.


The scaling of Section 4 is done in an analogous way as in Section 4, the corre-sponding SDEs are then expressed as

(46)

dXε(t) = −Xε(t−)∑i=1

N1/ε,i(dt) +Wε(t−)Nλ/ε(dt)−Xε(t−)∑i=1

Nβ/ε,i(dt),

dCε(t) = −Cε(t−)∑i=1

Nγ/ε,i(dt) + C1Nλ/ε(dt) + C2

Xε(t−)∑i=1

Nβ/ε,i(dt),

dΩε,a(t) = −αΩε,a(t) dt+ha(Cε(t)) dt, a∈p, d,

dWε(t) = −Wε(t−)∑i=1

Nδ,i(dt)+ApNΩε,p(t)(dt)−Ad1Wε(t−)≥AdNΩε,d(t)(dt),

Definition 4. For a fixed W=w, the fast variables of the SDEs (46) are associatedto a Markov process (Xw(t), Cw(t)) on N2 whose transition rates are given by, for(x, c)∈N2,

(x, c) −→

(x+w, c+C1) λ,

(x−1, c) x,−→

(x, c−1) γc,

(x−1, c+C2) βx.

λ 11+β

x

γc

x

c

+w

+C2

+C1

β1+β

x

Figure 2. Stochastic Queue for the Associated Fast Process ofthe Discrete Calcium-Based Model

This process can be seen as a network of two M/M/∞ queues with simultaneousarrivals, see Chapter 6 of Robert [44], as illustrated in Figure 2.

In the rest of this section, we will take C1=C2=1, to have a simpler setting. Theresults and proofs do not change, the explicit expression of the invariant distribu-tion, see Proposition 6, is more complicated.

The next result is the equivalent of Theorem 5 in a discrete setting. The maindifference is that there is a closed form expression, see Proposition 6 below, ofthe invariant distribution of fast variables. For these discrete models, an explicitrepresentation of the asymptotic dynamics of the synaptic weight is available inthis setting.

Theorem 6 (Averaging Principle for a Discrete Model). If hp and hd are func-tions on N with a finite range of values, the family of processes (Wε(t)) defined byRelations (46) is converging in distribution, as ε goes to 0, to the cadlag process(w(t)) satisfying the ODE

(47) dw(t) = −w(t−)∑i=1

Nδ,i(dt)+ApNωp(t)(dt)−Ad1w(t−)≥AdNωd(t)(dt),


and, for a∈p, d,

ωa(t) =

∫ t

0

e−α(t−s)∫N2

ha(c)ΠCQw(s)(dx, dc) ds,

where ΠCQw is the invariant distribution of the Markov process of Definition 4.

Note that for the functions for the calcium model with threshold functions,(hp, hd) defined by Relations (20) satisfy the conditions of the theorem. The proofis skipped. See the appendix of Robert and Vignoud [46]. Tightness properties areproved with standard arguments since the functions ha, a∈p, d are bounded, andthe discrete context simplifies some aspects, Harris irreducibility is not an issue inparticular.

Proposition 6 (Equilibrium of Fast Process). For w∈N, the Markov process onN2 of Definition 4 has a unique invariant distribution ΠCQ

w , and if the distributionof (Xw, Cw) is ΠCQ

w , the generating function of Cw is given by, for u∈[0, 1],

(48) E(uC

w)

= exp

(−λ∫ +∞

0

(1−(1−e−γs+ue−γs

)(1−p(s) + up(s))

w)ds

),

with

p(s) = β(e−γs−e−(β+1)s

)/(β+1−γ).

Only the distribution of the calcium variable Cw is considered due to its role inthe expression of (ωa(t)), a∈p, d in Theorem 6. The joint generating function of(Xw, Cw) can be obtained with the same approach.

It may be tempting to try to solve the equilibrium equations for the transitionrates of Definition 4. It does not seem that there is a way to solve them withgenerating functions methods. A probabilistic approach relying on a convenientrepresentation of the Markov process with a Poisson marked point process gives asatisfactory representation of the equilibrium distribution.

Proof. To each arrival instant t of the Poisson process Nλ on R is associated avector of N3w+1

u=((xi, 1≤i≤w), (yi, 1≤i≤w), (zi, 0≤i≤w)),

where, for 1≤i≤w,

(a) xi is the lifetime of the ith quantum of potential generated at time t,(b) z0, the lifetime of the quantum of calcium generated at t,(c) yi, the duration of time after which this ith quantum of potential initiates

a firing of the neuron,(d) zi, the lifetime of the ith quantum of calcium created if the event described

by (c) occurs.

We take (Un)=((Xn,i), (Yn,i), (Zn,i)), where (Xn,i), (Yn,i) and (Zn,i), sequences ofi.i.d. exponentially distributed random variables with respective parameters 1, βand γ, and independent of Nλ. Define

N λ(ds, du)def.=∑n∈Z

δ(tn,Un),


it is well known that N λ is a Poisson marked point process with intensity measure

(49) µ(ds,du)def.= λ ds

w⊗i=1

E1(dxi)

w⊗i=1

Eβ(dyi)

w⊗i=0

Eγ(dzi),

where Eξ(dx) is the exponential distribution with parameter ξ>0. See Chapter 5of Kingman [31] for example.

Assuming that Xw(0)=Cw(0)=0, with the interpretation of the coordinates ofthe mark u, it is easy to get the representation, for t≥0,

Xw(t) =

∫(0,t]

w∑i=1

1s+xi>t,s+yi>tN λ(ds, du),

Cw(t) =

∫(0,t]

1s+z0>tN λ(ds, du) +

∫(0,t]

w∑i=1

1xi>yi,s+yi<t,s+yi+zi>tN λ(ds, du).

By invariance by time-translation of the Poisson process N λ, we get that the ran-dom variable (Xw(t), Cw(t)) has the same distribution as

(Xw

(t), Cw(t))def.=

(∫(−t,0]

w∑i=1

1s+xi>0,s+yi>0N λ(ds, du) ,

∫(−t,0]

1s+z0>0N λ(ds, du) +

∫(−t,0]

w∑i=1

1xi>yi,s+yi<0,s+yi+zi>0N λ(ds, du)

).

The random variables (Xw

(t), Cw

(t)) are non-decreasing and converging to

(50) (Xw

(∞), Cw(∞))def.=

(∫(−∞,0]

w∑i=1

1s+xi>0,s+yi>0N λ(ds, du) ,

∫(−∞,0]

[1s+z0>0 +

w∑i=1

1xi>yi,s+yi<0,s+yi+zi>0

]N λ(ds, du)

).

The variable Xw

(∞) and Cw(∞) are almost surely finite since, with standardcalculations with Poisson processes, we obtain that

E[Xw

(∞)]

=λ

1+βw, E

[Cw(∞)

]=λ

γ

(1+

w

1+β

).

Recall the formula for Laplace transform of Poisson point processes,

E[exp

(∫−f(s, u)N λ(ds,du)

)]= exp

(∫ (1− e−f(s,u)

)µ(ds,du)

),

for any non-negative Borelian function f on R3w+2+ , where µ is defined by Rela-

tion (49). See Proposition 1.5 of Robert [44] for example. For u∈[0, 1], we therefore


get the relation

− lnE[uCw(∞)

]=∫

R+

(1−E

[u1Eγ,0>s

]E[u

∑wi=1 1E1,i>Eβ,i,Eβ,i<s<Eβ,i+Eγ,i

])λ ds =∫

R+

(1−(1−e−γs+ue−γs

)E[u1E1,1>Eβ,1,Eβ,1<s<Eβ,1+Eγ,1

]w)λ ds,

where (E1,i), (Eβ,i) and (Eγ,i) are independent i.i.d. exponentially distributedrandom variables with respective parameters 1, β and γ. Note that the p(s) of ourstatement is

p(s) = P(E1,1>Eβ,1, Eβ,1<s<Eβ,1+Eγ,1

),

the proposition is thus proved.

References

[1] A. Abbassian, M. Fotouhi, and M. Heidari. “Neural fields with fast learningdynamic kernel”. In: Biological cybernetics 106.1 (2012), pp. 15–26.

[2] A. E. Akil, R. Rosenbaum, and K. Josic. “Synaptic Plasticity in CorrelatedBalanced Networks”. In: bioRxiv (2020). eprint: https://doi.org/10.1101/2020.04.26.061515.

[3] B. Babadi and L. F. Abbott. “Stability and Competition in Multi-spike Mod-els of Spike-Timing Dependent Plasticity”. In: PLoS computational biology12.3 (Mar. 2016), e1004750.

[4] G.-q. Bi and M.-m. Poo. “Synaptic Modifications in Cultured HippocampalNeurons: Dependence on Spike Timing, Synaptic Strength, and PostsynapticCell Type”. In: Journal of Neuroscience 18.24 (Dec. 1998), pp. 10464–10472.

[5] P. Billingsley. Convergence of Probability Measures. John Wiley & Sons, 1999.[6] J. Chevallier, M. J. Caceres, M. Doumic, and P. Reynaud-Bouret. “Micro-

scopic approach of a time elapsed neural model”. In: Mathematical Modelsand Methods in Applied Sciences 25.14 (2015), pp. 2669–2719.

[7] E. Chichilnisky. “A simple white noise analysis of neuronal light responses”.In: Network: Computation in Neural Systems 12.2 (2001), pp. 199–213.

[8] C. Clopath, L. Busing, E. Vasilaki, and W. Gerstner. “Connectivity reflectscoding: a model of voltage-based STDP with homeostasis”. In: Nature Neu-roscience 13.3 (Mar. 2010), pp. 344–352.

[9] C. Clopath and W. Gerstner. “Voltage and Spike Timing Interact in STDP– A Unified Model”. In: Frontiers in Synaptic Neuroscience 2 (July 2010).

[10] M. H. A. Davis. Markov models and optimization. London: Chapman & Hall,1993.

[11] D. A. Dawson. “Measure-valued Markov processes”. In: Ecole d’Ete de Prob-abilites de Saint-Flour XXI—1991. Vol. 1541. Lecture Notes in Math. Berlin:Springer, Nov. 1993, pp. 1–260.

[12] S. N. Ethier and T. G. Kurtz. Markov Processes: Characterization and Con-vergence. John Wiley & Sons, Sept. 2009.

[13] C. W. Eurich, K. Pawelzik, U. Ernst, J. D. Cowan, and J. G. Milton. “Dy-namics of self-organized delay adaptation”. In: Physical Review Letters 82.7(1999), p. 1594.

https://doi.org/10.1101/2020.04.26.061515

https://doi.org/10.1101/2020.04.26.061515

REFERENCES 39

[14] D. E. Feldman. “The spike-timing dependence of plasticity”. In: Neuron 75.4(Aug. 2012), pp. 556–571.

[15] E. Fino, J. Glowinski, and L. Venance. “Bidirectional activity-dependent plas-ticity at corticostriatal synapses”. In: The Journal of Neuroscience: The Of-ficial Journal of the Society for Neuroscience 25.49 (Dec. 2005), pp. 11279–11287.

[16] M. I. Freidlin and A. D. Wentzell. Random perturbations of dynamical sys-tems. Second Edition. New York: Springer-Verlag, 1998.

[17] R. C. Froemke and Y. Dan. “Spike-timing-dependent synaptic modificationinduced by natural spike trains”. In: Nature 416.6879 (Mar. 2002), pp. 433–438.

[18] W. Gerstner and W. M. Kistler. Spiking Neuron Models: Single Neurons,Populations, Plasticity. Cambridge University Press, Aug. 2002.

[19] W. Gerstner, W. M. Kistler, R. Naud, and L. Paninski. Neuronal Dynamics:From Single Neurons to Networks and Models of Cognition. New York, NY,USA: Cambridge University Press, 2014.

[20] E. N. Gilbert and H. O. Pollak. “Amplitude Distribution of Shot Noise”. In:Bell System Technical Journal 39.2 (Mar. 1960), pp. 333–350.

[21] M. Graupner and N. Brunel. “Calcium-based plasticity model explains sen-sitivity of synaptic changes to spike pattern, rate, and dendritic location”.In: Proceedings of the National Academy of Sciences of the United States ofAmerica 109.10 (Mar. 2012), pp. 3991–3996.

[22] M. Graupner and N. Brunel. “Mechanisms of induction and maintenance ofspike-timing dependent plasticity in biophysical synapse models”. In: Fron-tiers in Computational Neuroscience 4 (2010).

[23] M. Graupner, P. Wallisch, and S. Ostojic. “Natural Firing Patterns ImplyLow Sensitivity of Synaptic Plasticity to Spike Timing Compared with FiringRate”. In: Journal of Neuroscience 36.44 (Nov. 2016), pp. 11238–11258.

[24] R. Gutig, R. Aharonov, S. Rotter, and H. Sompolinsky. “Learning input cor-relations through nonlinear temporally asymmetric Hebbian plasticity”. In:The Journal of Neuroscience: The Official Journal of the Society for Neuro-science 23.9 (May 2003), pp. 3697–3714.

[25] R. Z. Has’minskiı. Stochastic stability of differential equations. Alphen aanden Rijn: Sijthoff & Noordhoff, 1980, pp. xvi+344.

[26] P. Helson. “A new stochastic STDP Rule in a neural Network Model”. In:arXiv:1706.00364 [math] (Mar. 2018).

[27] M. J. Higley and B. L. Sabatini. “Calcium signaling in dendrites and spines:practical and functional considerations”. In: Neuron 59.6 (Sept. 2008),pp. 902–913.

[28] D. Horn, N. Levy, I. Meilijson, and E. Ruppin. “Distributed synchrony ofspiking neurons in a Hebbian cell assembly”. In: Advances in neural informa-tion processing systems. 2000, pp. 129–135.

[29] T. Kato. Perturbation theory for linear operators. Classics in Mathematics.Berlin: Springer-Verlag, 1995, pp. xxii+619.

[30] R. Kempter, W. Gerstner, and J. L. van Hemmen. “Hebbian learning andspiking neurons”. In: Physical Review E 59.4 (Apr. 1999), pp. 4498–4514.

[31] J. F. C. Kingman. Poisson Processes. Clarendon Press, Dec. 1992.

40 REFERENCES

[32] W. M. Kistler and J. L. v. Hemmen. “Modeling Synaptic Plasticity in Con-junction with the Timing of Pre- and Postsynaptic Action Potentials”. In:Neural Computation 12.2 (Feb. 2000), pp. 385–405.

[33] T. G. Kurtz. “Averaging for martingale problems and stochastic approxi-mation”. In: Applied Stochastic Analysis. Ed. by I. Karatzas and D. Ocone.Vol. 177. Berlin/Heidelberg: Springer-Verlag, 1992, pp. 186–209.

[34] L. Lapicque. “Recherches Quantitatives sur l’Excitation Electrique des NerfsTraitee comme une Polarization”. In: Journal de Physiologie et PathologieGeneral 9 (1907), pp. 620–635.

[35] E. V. Lubenov and A. G. Siapas. “Decoupling through synchrony in neuronalcircuits with propagation delays”. In: Neuron 58.1 (Apr. 2008), pp. 118–131.

[36] A. Morrison, A. Aertsen, and M. Diesmann. “Spike-timing-dependent plastic-ity in balanced random networks”. In: Neural Computation 19.6 (June 2007),pp. 1437–1467.

[37] A. Morrison, M. Diesmann, and W. Gerstner. “Phenomenological models ofsynaptic plasticity based on spike timing”. In: Biological Cybernetics 98.6(June 2008), pp. 459–478.

[38] J. Nakai, M. Ohkura, and K. Imoto. “A high signal-to-noise Ca(2+) probecomposed of a single green fluorescent protein”. In: Nature Biotechnology 19.2(Feb. 2001), pp. 137–141.

[39] G. Papanicolalou, D. W. Stroock, and S. R. S. Varadhan. “Martingale ap-proach to some limit theorems”. In: Proc. 1976. Duke Conf. On Turbulence.III. Duke Univ. Math, 1977.

[40] R. Pawula. “Generalizations and extensions of the Fokker-Planck-Kolmogorovequations”. In: IEEE Transactions on Information Theory 13.1 (1967),pp. 33–41.

[41] B. Perthame, D. Salort, and G. Wainrib. “Distributed synaptic weights in aLIF neural network and learning rules”. In: Physica D: Nonlinear Phenomena353 (2017), pp. 20–30.

[42] J.-P. Pfister and W. Gerstner. “Triplets of Spikes in a Model of SpikeTiming-Dependent Plasticity”. In: Journal of Neuroscience 26.38 (Sept.2006), pp. 9673–9682.

[43] N. Ravid Tannenbaum and Y. Burak. “Shaping Neural Circuits by High OrderSynaptic Interactions”. In: PLoS computational biology 12.8 (2016), e1005056.

[44] P. Robert. Stochastic Networks and Queues. Stochastic Modelling and Ap-plied Probability. Berlin Heidelberg: Springer-Verlag, 2003.

[45] P. Robert and J. Touboul. “On the Dynamics of Random Neuronal Net-works”. In: Journal of Statistical Physics 165.3 (Nov. 2016), pp. 545–584.

[46] P. Robert and G. Vignoud. “Mathematical Models of Neural Plasticity: Av-eraging Principles”. Oct. 2020.

[47] P. D. Roberts. “Computational Consequences of Temporally AsymmetricLearning Rules: I. Differential Hebbian Learning”. In: Journal of Compu-tational Neuroscience 7.3 (Nov. 1999), pp. 235–246.

[48] B. S. Robinson, T. W. Berger, and D. Song. “Identification of Stable Spike-Timing-Dependent Plasticity from Spiking Activity with Generalized Multi-linear Modeling”. In: Neural Comput. 28.11 (Nov. 2016), pp. 2320–2351.

[49] M. C. van Rossum, G. Q. Bi, and G. G. Turrigiano. “Stable Hebbian learningfrom spike timing-dependent plasticity”. In: The Journal of Neuroscience:

REFERENCES 41

The Official Journal of the Society for Neuroscience 20.23 (Dec. 2000),pp. 8812–8821.

[50] J. Rubin, D. D. Lee, and H. Sompolinsky. “Equilibrium properties of tempo-rally asymmetric Hebbian plasticity”. In: Physical Review Letters 86.2 (Jan.2001), pp. 364–367.

[51] J. Rubin, B. Krauskopf, and H. Osinga. “Natural extension of fast-slowdecomposition for dynamical systems”. In: Physical Review E 97.1 (2018),p. 012215.

[52] C. J. Shatz. “The Developing Brain”. In: Scientific American 267 (1992),pp. 60–67.

[53] H. Z. Shouval, M. F. Bear, and L. N. Cooper. “A unified model of NMDAreceptor-dependent bidirectional synaptic plasticity”. In: Proceedings of theNational Academy of Sciences of the United States of America 99.16 (Aug.2002), pp. 10831–10836.

[54] T. Takeuchi, A. J. Duszkiewicz, and R. G. M. Morris. “The synaptic plasticityand memory hypothesis: encoding, storage and persistence”. In: PhilosophicalTransactions of the Royal Society of London. Series B, Biological Sciences369.1633 (Jan. 2014), p. 20130288.

[55] S. Tonegawa, M. D. Morrissey, and T. Kitamura. “The role of engram cellsin the systems consolidation of memory”. In: Nature Reviews Neuroscience19.8 (Aug. 2018), pp. 485–498.

[56] G. G. Turrigiano and S. B. Nelson. “Homeostatic plasticity in the developingnervous system”. In: Nature Reviews. Neuroscience 5.2 (Feb. 2004), pp. 97–107.

[57] G. Vignoud, L. Venance, and J. D. Touboul. “Interplay of multiple pathwaysand activity-dependent rules in STDP”. In: PLoS computational biology 14.8(2018), e1006184.

[58] R. S. Zucker and W. G. Regehr. “Short-term synaptic plasticity”. In: AnnualReview of Physiology 64 (2002), pp. 355–405.

42 REFERENCES

Appendix A. Discussion of Some Aspects of STDP Models

A.1. Plasticity Models Without Exponential Filtering. In the model of Sec-tion 2, with Relation (8) we defined a filtering procedure with an exponential kernelof rate α>0 for the function Ωa, where Ωp(t) and Ωd(t) are used to quantify thepast activity of input and output neurons leading to potentiation and depressionrespectively. It is given by, for a∈p, d,

dΩa(t) = −αΩa(t) dt+ Γa(Nλ,Nβ,X)(dt),

where Γa(Nλ,Nβ,X)(dt) represents the differential element for the instantaneoussynaptic plastic processes for potentiation, a=p, and, for depression, a=d.

Therefore, the update of the synaptic weight at time t depends on a functional ofthe synaptic processes that happened before t. The dynamic of the synaptic weight(W (t)) is defined by,

dW (t) = M (Ωp(t),Ωd(t),W (t)) dt,

Several studies of computational neuroscience have investigated the role of STDPin a stochastic setting. See Kempter et al. [30], Kistler and Hemmen [32], Roberts[47], Rubin et al. [50], and Morrison et al. [37] for example. These references usemore “direct” dynamics for the synaptic weight. The update at time t dependsonly on the current synaptic plastic processes Γa(Nλ,Nβ,X)(dt) at time t, insteadof a smoothed version over the past activity. The associated model can be definedso that the corresponding synaptic weight process (W (t)) satisfies the relation

dW (t) = M(

Γp(Nλ,Nβ,X),Γd(Nλ,Nβ,X),W (t))

(dt),

for some functional M .

Biological Arguments For Exponential Filtering. It should be noted that the modelassociated to (W (t)) does not seem to be in agreement with observations of numer-ous experimental studies. See Bi and Poo [4], Fino et al. [15], and Feldman [14]. Ina classical experiment, the protocol to induce plasticity consists in stimulating bothneurons at a certain frequency a fixed number of times with a fixed delay ∆t, overa period of up to one or two minutes (60-100 pairings at 1 Hz for example). Thispart is designed to reproduce conditions of correlations between the two neurons,when mechanisms of plasticity are known to be triggered. However, measurementsof the synaptic weight show that changes take place on a different timescale. Afterthe end of the protocol, it is observed that at least several minutes are necessaryto have a significant and stable effect on the synaptic weight. In other words, thechange in synaptic weights happens long after the end of the plasticity induction.

For this reason we have chosen to use a filter, possibly with an exponentialkernel, on the past synaptic activity. Therefore it does not only depend on theinstantaneous synaptic variable Γd(Nλ,Nβ,X)(dt) at time t, but on the whole pastΓd(Nλ,Nβ,X)(ds), s≤t, with a smoothing exponential kernel which gives the desireddynamical feature for the synaptic weight. Another recent article Robinson et al.[48] also takes this fact into account by adding an “ınduction” function to theclassical models of STDP.

We have chosen to use this more biological hypothesis in our model. We statethe analogous stochastic averaging principles for the model without an exponentialfiltering.

appendix 43

Definition 5 (Scaled Dynamical System for Instantaneous Plasticity). We definethe stochastic process (X(t),W (t)) with initial state (x0, w0), satisfying the evolu-tion equations, for t>0,

(51)

dXε(t) = −1

εXε(t) dt+Wε(t−)Nλ/ε(dt)− g(Xε(t−))Nβ/ε,Xε(dt),

dW ε(t) = εM(

Γp(Nλ/ε,Nβ/ε,Xε),Γd(Nλ,Nβ,X),W (t))

(dt),

where Γp and Γd are plasticity kernels. The functional M is defined by

M : M+(R+)2×R 7→ M+(R+)(52)

(Γp,Γd, w)→M(Γp,Γd, w).

We have to modify Assumptions B-(b) by Assumptions B*-(b), in the followingway, M can be decomposed as, M(Γp,Γd, w)=Mp(w)Γp−Md(w)Γd − δw, where

Ma(w) is non-negative continuous function, and,

Ma(w) ≤ CM ,for all w∈KW , for a∈p, d.

An analogue of Theorem 2 for plasticity kernels of class (M) in this context isthe following result.

Theorem 7 (Averaging Principle for Instantaneous Plasticity). Under Assump-tions A and B* and for initial conditions satisfying Relation (31), there existsS0∈(0,+∞], such that the family of processes (W ε(t), t<S0) associated to Rela-tions (51) and (52), is tight for the convergence in distribution as ε goes to 0.Almost surely, any limiting point (w(t), t<S0) satisfies the relation(53)

dw(t) =

∫R×R`+M( [

(na,0(z)+λna,1(z)+β(x)na,2(z)) dt]a∈p,d , w(t)

)Πw(t)(dx, dz).

where, for w∈KW , Πw is the invariant measure Πw of the operator BFw of Rela-tion (33).

Proof. Due to the specific expression of M , the arguments follow the ones usedin Robert and Vignoud [46]. The proof is skipped.

Comparison with Theorem 2. Both theorems show that the dynamics of thesynaptic weight w in the decoupled stochastic system depend on an integral over thestationary distribution of the fast process. However, in Theorem 2, the averagingproperty occurs at the level of the synaptic plasticity processes ωa,

dωa(t)

dt= −αωa(t) +

∫R×R`+

[na,0(z)+λna,1(z)+β(x)na,2(z)

]dΠw(t)(x, z),

and, the function M is applied afterwards to have the update of the synaptic weightw,

dw(t)

dt= M(ωp(t), ωd(t), w(t)).

In Theorem 7, with no exponential filtering, the averaging is applied directly at thelevel of the synaptic update,

dw(t) =

∫R×R`+M( [

(na,0(z)+λna,1(z)+β(x)na,2(z)) dt]a∈p,d , w(t)

)Πw(t)(dx, dz).

44 appendix

A Toy Example. We defineM(ωp, ωd, w) = ωp−ωd, (ωp, ωd, w)∈R2

+×R,M(Γ1,Γ2, w) = Γ1−Γ2, Γ1,Γ2∈M+(R+),

Γp(dt)−Γd(dt) = (F−W (t)) dt,

with F>0. The equations for the time evolution of synaptic weights are thus givenby

dW (t)

dt= ε

(F−W (t)

)and

dW (t)

dt= α2

∫ t

0

e−α(t−s)(F−W (s)) ds,

with the initial condition W (0)=W (0)=w0>0. We get that

W (t) = F+(w0−F )e−εt, t≥0,

so that (W (t)) converges to F as t gets large, as it can be expected. By differenti-ating the relation for (W (t)) we obtain,

d2W (t)

dt2+ α

dW (t)

dt+W (t) = F,

with W (0)=w0 and W ′(0)=0. If we take α=2ε with ε<1, we get that

W (t) = F + (w0−F )e−εt

(cos(t√

1−ε2)

+

√ε2

1− ε2sin(t√

1−ε2))

,

in particular ((W (t)−W (t))eεt/(w0−F )) is a periodic function with maximal valueof the order of 1/ε. Both functions (W (t)) and (W (t)) converge to F as t goes toinfinity at the same exponential rate but differ at the second order.

A comparison of both models is also done in Section B of the Appendix andillustrated for pair-based rules in Figure 3 and for calcium-based ones in Figure 4.

appendix 45

A.2. Links with Models of Physics. In this section, averaging principles for all-to-all pair-based STDP rules of Kempter et al. [30] are discussed. After adaptingnotations, the main equation, Relation (4) of this reference, for the asymptoticbehavior of the time evolution of the synaptic weight, is expressed, via a separationof time scales argument, as the solution of an ODE

(54)dw

dt= w1ν1(t) + w2ν2(t) +

∫ +∞

−∞Φ(s)µ(s, t) ds,

where,

— w1=Dp,1−Dd,1 and ν1(t)=λ (in our case, the input rate is constant);— w2=Dp,2−Dd,2 and

ν2(t)=

∫R+

β(x)ΠPAw(t)(dx),

where ΠPAw is defined in Proposition 3, is the firing rate of the post-synaptic

neuron at equilibrium when the synaptic weight is w(t);

— Φ(t)=Φp(t)−Φd(t), with

Φa(t) = Ba,1 exp(−γa,1t)1t>0 +Ba,2 exp(γa,2t)1t<0.

— µ(s, t)=<S1(t+s)S2(t)>, where S1, resp. S2, is the process of pre-synapticspikes, resp. post-synaptic spikes.

— The quantity 〈· · ·〉 is defined in terms of temporal and ensemble averagesthat are not completely clear from a mathematical point of view, <· · ·> isthe ensemble average and · · · the temporal average over the spike trains.See Kempter et al. [30].

Our guess is that the quantity µ(s, t) can in fact be interpreted asµ(w(s), t), with

µ(w, t) =

limh0

EΠPAw

(Nλ[0, h]Nβ,X [t, t+h])

h2, for t<0;

limh0

EΠPAw

(Nλ[0, h]Nβ,X [−t,−t+h])

h2, for t<0.

The limits, related to second order properties of the point processes Nλ andNβ,X , are assumed to hold.

Our approach in this section is essentially heuristic. The following, non-rigorous,derivations are done to establish a connection with our main results in this specificcase. For w∈KW , from the definition of Φa, a∈p, d,∫ 0

−∞Φa(s)EΠPA

w

(Nβ,X [0, h]

h

Nλ[s, s+h]

h

)ds

= Ba,2EΠPAw

(∫ 0

−∞exp(γa,2s)EΠPA

w

[Nβ,X [0, h]

h

Nλ[s, s+ h]

h

∣∣∣∣F0

]ds

)= Ba,2EΠPA

w

(EΠPA

w

[Nβ,X [0, h]

h

∣∣∣∣F0

] ∫ −h−∞

exp(γa,2s)Nλ[s, s+ h]

hds

)

∼ Ba,2EΠPAw

(β(X(0))

∫ −h−∞

exp(γa,2s)Nλ[s, s+ h]

hds

)

46 appendix

and, if Nλ=(tn, n∈Z), with t0≤0<t1,

= Ba,2EΠPAw

β(X(0))∑

tn≤−h

1

h

∫ tn+h

tn

exp(γa,2s) ds

∼ Ba,2

γa,2EΠPA

w

β(X(0))∑n≤0

exp(γa,2tn)

=Ba,2γa,2

EΠPAw

(β(X(0))Za,2(0)) =Ba,2γa,2

∫R5

+

β(x)za,2ΠPAw (dx, dz)

by using a representation of Za,2(0) similar to that of Xw∞(0) with Relation (38).

Similarly,∫ +∞

0

Φa(s)EΠPAw

(Nβ,X [0, h]

h

Nλ[s, s+h]

h

)ds ∼ Ba,1

γa,1

∫R5

+

β(x)za,1ΠPAw (dx, dz).

Recall that the model of Kempter et al. [30] is without exponential filtering, seeSection A.1. With this interpretation of µ, Relation (54) can thus be rewrit-ten in our setting, as Relation (53), with M(Γp,Γd, w)=Γp−Γd, and na,0(z)=0,na,1(z)=Da,1+za,2 and na,2(z)=Da,2+za,1, where za,i are defined as in Assump-tions PA.

Extensions. The interest of Relation (54) is that it may be formulated for a generalplasticity curve Φa for all-to-all pair-based models. Recall that the correspondingplasticity kernels are of classM only for exponential functions. We conjecture thatunder the conditions, for a∈p, d,

—

∫ +∞

−∞|Φa(s)|ds < +∞;

— limt→0+

Φa(t) and limt→0−

Φa(t) exist,

the convergence of the scaled process to the ODE (54) with a convenient µ shouldhold. For plasticity kernels of class M, this is done by using Markov properties ofthe fast processes (Xε(t), Zε(t)). See Robert and Vignoud [46]. We do not havethis tool in the case of a general plasticity curve. The proof of such an extensionshould require an additional analysis.

Appendix B. Graphical Representation of Models of Plasticity

In this section, we will consider several examples of simple dynamics of theMarkovian defined in Section 3.

We will start by comparing the effect of three different Hebbian pair-based rules,both on model with, Section 3, and without, Section A.1, exponential filtering.Then, we will focus on calcium-based models and show that the discrete model ofSection 7 can be a good approximation of the continuous model of Section 6.

We consider two different timescales to compare the induction of plasticity inmodel with or without exponential filtering,

— A fast timescale, on the order of the membrane potential dynamics (seeplain black line under each row), where the input and output spike patternsare presented.

appendix 47

— A slow timescale (20 times slower in this example), on the order of thesynaptic weight modifications (see dotted black line), where no input ispresented.

Input and output spikes patterns are fixed in both Figures (see first row).

B.1. Pair-based STDP Rules (Figure 3). In this section, we describe the dy-namics of the different stochastic processes involved in the pair-based STDP model.

In particular, we compare the various interpretation of the pair-based rules thatare described in 2.4.1 in Figure 3,

(a) All-to-all Model,(b) Nearest Neighbor Symmetric Model,(c) Nearest Neighbor Symmetric Reduced Model.

The different interactions are represented by Grey arrows (first row).Exponential STDP curves are considered with their associated Markovian de-

scription, see Section C.1.Finally, we focus on Hebbian STDP rules with Bd,2=0 and Bp,1=0.In the second row, the time evolution of the membrane potential,

dX(t) = −X(t) dt+W (t−)Nλ(dt)−X(t−)Nβ,X(dt),

is presented. Two interesting facts are to be noted here, at each pre-synaptic spike(green, first row), the current value of the synaptic weight W (t−) is added to themembrane potential X(t). It can be seen in this example that the size of the jumpvaries across time. In addition, a complete reset of X occurs after a post-synapticspike (purple, first row), corresponding to g(x)=x.

Then we focus on the instantaneous synaptic variables Zp,1 (brown, third row)and Zd,2 (brown, fourth row), that follows different dynamics depending on the rulechosen.

(a) For all-to-all pairings, each synaptic spike is paired with all previous post-synaptic spikes, and conversely. They are already described in the maintext, by the set of equations, for a∈p, d,

dZp,1(t) = −γp,1Zp,1(t) dt+Bp,1Nλ(dt),

dZd,2(t) = −γd,2Zd,2(t) dt+Bd,2Nβ,X(dt).

All pairs of pre-synaptic and post-synaptic spikes are taken into account.(b) For nearest neighbor symmetric scheme, each pre-synaptic spike is paired

with the last post-synaptic spike, and conversely, the system changesslightly:

dZp,1(t) = −γp,1Zp,1(t) dt+(Bp,1−Zp,1(t−))Nλ(dt),

dZd,2(t) = −γd,2Zd,2(t) dt+(Bd,2−Zd,2(t−))Nβ,X(dt).

The variable Zp,1, resp. Zd,2 is reset to Bp,1, resp. Bd,2, after a pre-synapticspike, resp. post-synaptic spike.

(c) For nearest neighbor symmetric reduced scheme, where only immediate pair-ing matters, we have:dZp,1(t) = −γp,1Zp,1(t) dt+(Bp,1−Zp,1(t−))Nλ(dt)−Zp,1(t−)Nβ,X(dt),

dZd,2(t) = −γd,2Zd,2(t) dt+(Bd,2−Zd,2(t−))Nβ,X(dt)−Zd,2(t−)Nλ(dt),

48 appendix

Outp

ut

Nβ,X

t post

Input

Nλ

t pre

Mem

bra

ne

Pote

nti

alX

Pote

nti

ati

on

Γp

=Zp,1Nβ,X

Dep

ress

ion

Γd

=Zd,2Nλ

Slo

wP

last

icit

yV

ari

able

sΩa

Synapti

cW

eightW

Iterations

Toy

Synapti

c

Wei

ghtW

Iterations Iterations

(a) Hebbian All-to-all

Pair-Based RulesSection 3.3

(b) Hebbian Nearest

Neighbor SymmetricPair-Based Rules

Section 3.4

(c) Hebbian Nearest

Neigbhor Symmetric ReducedPair-Based Rules

Section 3.4

Figure 3. Synaptic Plasticity Kernels for Pair-Based Rules

appendix 49

The variable Zp,1 is reset to Bp,1, after a pre-synaptic spike and to 0 aftera post-synaptic spike, and conversely for Zd,2.

This simple example shows how different pair-based rules shape the instantaneousplasticity variables Z. This dependence is subsequently transferred to the poten-tiation kernel Γp (red, third row) and the depression kernel Γd (blue, fourth row).With exponential pair-based models, we have na,0(z)=0, nd,1(z)=zd,2, np,1(z)=0,nd,2(z)=0, np,2(z)=zp,1, and therefore, they follow,

Γp(dt) = Zp,1(t−)Nβ,X(dt)

Γd(dt) = Zd,2(t−)Nλ(dt).

It is then not surprising to observe that for a same sequence of pre- and post-synaptic spikes the plasticity kernels are different.

Consequently, it is the same for the slow plasticity variables Ωp (red, fifth row)and Ωd (blue, fifth row), that follows,

dΩp(t) = −αΩp(t) dt+ Zp,1(t−)Nβ,X(dt)

dΩd(t) = −αΩd(t) dt+ Zd,2(t−)Nλ(dt),

We choose in this example a linear function M , leading to the following timeevolution of the synaptic weight (sixth row),

dW (t) = (Ωp(t)− Ωd(t)) dt.

This example shows that a simple change in the STDP rule can lead to very dif-ferent dynamics for the synaptic weight. All-to-all rules lead to global potentiation(the dotted line represents the initial value) whereas nearest neighbor rules lead todepression.

Finally, as can be expected from the slow plasticity variables Ωa that are stillpositive long after the end of the stimulus (see in the dotted part), the synapticweight is modified long after the patterns of spikes.

On the contrary, considering the model without exponential filtering (seventhrow),

dW (t) = Γp(dt)− Γd(dt),

we see that in that case, the synaptic weight is only updated during the stimulus.We notice that the polarity of the global plasticity is the same as with exponentialfiltering, but the dynamics are completely different, as showed with the toy modelin Section A.1.

B.2. Calcium-based STDP Rules (Figure 4). In this section, we focus on thedynamics of the calcium-based models,

(a) the continuous version, described in Section 6;(b) the discrete version from Section 7.

The continuous membrane potential (second row, left) follows,

dX(t) = −X(t) dt+W (t−)Nλ(dt)−Nβ,X .We consider a different function g(x)=1 than in the previous case. Its discreteanalogue (second row, right) verifies,

dX(t) = −X(t−)∑i=1

N1,i(dt) +W (t−)Nλ(dt)−X(t−)∑i=1

Nβ,i(dt).

50 appendix

Input

Nλ

t pre

Outp

ut

Nβ,X

t post

Mem

bra

ne

Pote

nti

alX

Calc

ium

Conce

ntr

ati

onC

θp

θd

Pla

stic

ity

Ker

nel

sΓa

Slo

wP

last

icit

yV

ari

able

sΩa

Synapti

cW

eightW

Iterations

Toy

Synapti

c

Wei

ghtW

Iterations

(a) Continuous

Calcium-Based RulesSection 3.1

(b) Stochastic Queue

Calcium-Based RulesSection 7

Figure 4. Synaptic Plasticity Kernels for Calcium-Based Rules

appendix 51

It is plainly clear that both processes are almost identical, except that the ex-ponential decay in the continuous model is replaced by a M/M/∞ queue in thediscrete case. In the case of large jumps, they lead to a similar dynamical evolution.

The same conclusions can be drawn for the calcium concentration, where thecontinuous version (third row, left) follows,

dC(t) = −γC(t) dt+C1Nλ(dt)+C2Nβ,X(dt),

and the discrete version (third row, right),

dC(t) = −C(t−)∑i=1

Nγ,i(dt) + C1Nλ(dt) + C2

X(t−)∑i=1

Nβ,i(dt).

In both cases, the plasticity kernels Γp (fourth row, red) and Γd (fourth row,blue) verify,

Γp(dt) = 1C(t−)≥θp dt

Γd(dt) = 1C(t−)≥θd dt.

When the calcium reaches the thresholds θp for potentiation (third row, red)and θd (third row, blue), the plasticity kernels are “activated” and are equal todt. We see that both models leads to similar values of the kernels, even if somediscrepancies start to appear.

The slow plasticity variables (fifth row) are just obtained by integration of thekernels with an exponential filtering,

dΩp(t) = −αΩp(t) dt+ Γp(dt) dt

dΩd(t) = −αΩd(t) dt+ Γd(dt) dt.

A second discretization is applied in the synaptic update, the continuous version(sixth row, left) verifies,

dW (t) =(ApΩp(t)−Ad1W (t−)≥0Ωd(t)

)dt,

and the discrete one (sixth row, right),

dW (t) = ApNΩp(t−)(dt)−Ad1W (t−)≥0NΩd(t−)(dt).

We note here that we need to force W to stay non-negative in order to have a validdescription of the system. We observe that, even after two different discretizations,both synaptic weights follow a similar evolution.

Using a model without exponential filtering (seventh row) leads to a differentdynamical evolution of the synaptic weight, for the continuous model,

dW (t) = ApΓp(dt)−Ad1W (t−)≥0Γd(dt),

and the discrete one,

dW (t) = ApN1C(t−)≥θp(dt)−Ad1W (t−)≥AdN1C(t−)≥θd

(dt).

As a conclusion, the discrete models approximate well the continuous one andtherefore, using the exact expressions of the discrete model can give an interestinginsight on the dynamics of the continuous model.

52 appendix

Appendix C. Operators of Fast Systems of STDP models

We first start with the generator of a general STDP of class M as in Defini-tion 3. For u=(x, z, ωp, ωd, w)=(ui, 1≤i≤`+4)∈SM(`) and f∈C1

b (SM(`)), i.e. f isa bounded C1-function, and all its respective derivatives are bounded, by usingEquations (30), it is not difficult to show that the extended infinitesimal generatorA of (U(t)) can be expressed as,

A(f)(u) = (−αωp+np,0(z))∂f

∂ωp(u)+ (−αωd+nd,0(z))

∂f

∂ωd(u)

−x∂f∂x

(u)+

⟨−γz+k0,

∂f

∂z(u)

⟩+M(ωp, ωd, w)

∂f

∂w(u)

+λ

[f(u+we1+k1(z)e2+np,1(z)e`+2+nd,1(z)e`+3

)−f (u)

]+β(x)

[f(u−g(x)e1+k2(z)e2+np,1(z)e`+2+nd,1(z)e`+3

)−f(u)

]with the following notations, ei is the unit vectors for the coordinates with index i.The notation (

∂f

∂z(u)

)def.=

(∂f

∂ui(u), 2≤i≤`+1

)is for the gradient vector with respect to the coordinates associated to z, i.e. fromindex 2 to index `+1. Finally e2 is the vector whose coordinates are 1 for theindices associated to z and 0 elsewhere and, for a∈R`+, the quantity ae2 is thevector whose ith coordinate is ai−1, for 2≤i≤`+1, and 0 otherwise.

For sake of completeness, in the context of Section 4, we give a list of the in-finitesimal generators of the fast variables of the classical STDP rules described inSection 2.4.

C.1. Pair-Based Models with Exponential Kernels Φ. For pair-based mech-anisms, we follow the classification discussed in Morrison et al. [37]:

— For all-to-all pairings, each synaptic spike is paired with all previous post-synaptic spikes, and conversely. They are already described in the maintext, by the set of equations, for a∈p, d,

dX(t) = −X(t) dt+wNλ(dt)−g (X(t−))Nβ,X (dt) ,

dZa,1(t) = −γa,1Za,1(t) dt+Ba,1Nλ(dt),

dZa,2(t) = −γa,2Za,2(t) dt+Ba,2Nβ,X(dt).

— For nearest neighbor symmetric scheme where each pre-synaptic spike ispaired with the last post-synaptic spike, and conversely, the system changesslightly:


dZa,1(t) = −γa,1Za,1(t) dt+(Ba,1−Za,1(t−))Nλ(dt),

dZa,2(t) = −γa,2Za,2(t) dt+(Ba,2−Za,2(t−))Nβ,X(dt).

The variable Za,1, resp. Za,2 is reset to Ba,1, resp. Ba,2, after a pre-synapticspike, resp. post-synaptic spike.

appendix 53

— For nearest neighbor pre-synaptic centered scheme where each synaptic spikeis paired with the previous one and the following post-synaptic spike, thesystem is:dX(t) = −X(t) dt+wNλ(dt)−g (X(t−))Nβ,X (dt) ,

dZa,1(t) = −γa,1Za,1(t) dt+(Ba,1−Za,1(t−))Nλ(dt),

dZa,2(t) = −γa,2Za,2(t) dt+(Ba,2−Za,2(t−))Nβ,X(dt)−Za,2(t−)Nλ(dt),

— For nearest neighbor symmetric reduced scheme, where only immediate pair-ing matters, we have:dX(t) = −X(t) dt+wNλ(dt)−g (X(t−))Nβ,X (dt) ,

dZa,1(t) = −γa,1Za,1(t) dt+(Ba,1−Za,1(t−))Nλ(dt)−Za,1(t−)Nβ,X(dt),

dZa,2(t) = −γa,2Za,2(t) dt+(Ba,2−Za,2(t−))Nβ,X(dt)−Za,2(t−)Nλ(dt),

for exponential pair-based models, with na,0(z)=0, na,1(z)=za,2 and na,2(z)=za,1.

C.2. Nearest Pair-Based Models with General Kernels Φ. In the case ofnearest pair-based models, we have a simple description of the system, based on thetime since the last spike as detailed in Section 3.4. We define (Z(t))=(Z1(t), Z2(t)),such that,

dZ1(t) = dt−Z1(t−)Nλ(dt),

dZ2(t) = dt−Z2(t−)Nβ,X(dt).

In this setting, both nearest models are of class M:

— The nearest neighbor symmetric model of Relation (15), with

na,0(z)=0, na,1(z)=Φa,2(z2), na,2(z)=Φa,1(z1).

— The nearest neighbor symmetric reduced model of Relation (17), with

na,0(z)=0, na,1(z)=Φa,2(z2)1z2≤z1, na,2(z)=Φa,1(z1)1z1≤z2.

In fact, we have here two different Markovian systems that represents the samedynamics for nearest exponential STDP rules.

C.3. Triplet-Based Models. Generator for triplet-based mechanisms can also bedefined in a similar way, see Babadi and Abbott [3] for a list of different implemen-tations.

— The suppression model of Section 2.4.3 from Froemke and Dan [17], wherethe Markovian system is given by:


dZa,1(t) = −γa,1Za,1(t) dt+Ba,1Nλ(dt),

dZa,2(t) = −γa,2Za,2(t) dt+Ba,2Nβ,X(dt),

dZs,1(t) = −δ1(Zs,1(t)−1) dt−Zs,1(t−)Nλ(dt),

dZs,2(t) = −δ2(Zs,2(t)−1) dt−Zs,2(t−)Nβ,X(dt),

with na,0(z)=0, na,1(z)=zs,1zs,2za,2 and na,2(z)=zs,1zs,2za,1.

54 appendix

— The triplet-based model, see Pfister and Gerstner [42], we have:


dZa,1(t) = −γa,1Za,1(t) dt+(Ba,1+Zs,a,1(t−))Nλ(dt),

dZa,2(t) = −γa,2Za,2(t) dt+(Ba,2+Zs,a,2(t−))Nβ,X(dt),

dZs,a,1(t) = −δa,1Zs,a,1(t) dt+Da,1Nλ(dt),

dZs,a,2(t) = −δa,2Zs,a,2(t) dt+Da,2Nβ,X(dt),

with na,0(z)=0, na,1(z)=za,2 and na,2(z)=za,1. In this case, ki are notbounded and therefore this model does not verify Assumptions B.

C.4. Calcium-Based Models. For models of calcium-based plasticity, we have:

— Calcium transients as exponential traces in Graupner and Brunel [21], whichis the dynamics used as an example in this paper. The system is,

dX(t) = −X(t) dt+wNλ(dt)−g(X(t−))Nβ,X(dt),

dC(t) = −γC(t) dt+C1Nλ(dt)+C2Nβ,X(dt).

— Calcium transients modeled in a discrete setting as for the example in Sec-tion 7. The associated Markov process has the following transitions transi-tion rates, for (x, c)∈N2,

(x, c) −→

(x+w, c+1) λ,

(x−1, c) x,−→

(x, c−1) γc,

(x−1, c+1) βx.

The functions of calcium-based models are given by, for a∈p, d, na,0(c)=ha(c),na,1(x, c)=0 and na,2(c)=0.

C.5. Voltage-Based Models. Models of Section 2.4.5, which are adaptationsof Clopath and Gerstner [9] by replacing the direct dependence on filtered tracesof X, can also be analyzed with this formalism. The dynamics are given by


dZp,1(t) = −γp,1Zp,1(t) dt+Nλ(dt),

dZa,2(t) = −γa,2Za,2(t) dt+Ba,2Nβ,X(dt),

with na,0(z)=0, np,1(z)=0, np,2(z)=zp,1(zp,2−θd)+, nd,1(z)=(zd,2−θd)+, nd,2(z)=0.Email address: [email protected]

URL: http://www-rocq.inria.fr/who/Philippe.Robert

(Ph. Robert, G. Vignoud) INRIA Paris, 2 rue Simone Iff, 75589 Paris Cedex 12, France

Email address: [email protected]

(G. Vignoud) Center for Interdisciplinary Research in Biology (CIRB) - College deFrance (CNRS UMR 7241, INSERM U1050), 11 Place Marcelin Berthelot, 75005 Paris,

France

Documents

Abstract. arXiv:2010.08195v1 [q-bio.NC] 16 Oct 2020Abstract. In neuroscience, learning and memory are usually associated to long-term changes of connection strength between neurons