Classification: Biological Sciences, Neuroscience

Title: A triplet model of spike-timing dependent plasticity generalizes the BCM rule to spatio-temporal inputs

Authors: Julijana Gjorgjieva1, Claudia Clopath2, Juliette Audet3, Jean-Pascal Pfister4,5

Address: (1) Department of Applied Mathematics and Theoretical Physics, University of Cam-bridge, Cambridge, UK(2) Laboratory of Neurophysics and Physiology, Université Paris Descartes, Paris, France(3) Laboratory of Computational Neuroscience, Ecole Polytechnique Fédérale de Lausanne,Lausanne, Switzerland(4) Computational Neuroscience Lab, Physiology Department, University of Bern, Bern, CH(5) Department of Engineering, University of Cambridge, Cambridge, UK

Corresponding author: Julijana Gjorgjieva

Corresponding author address: Department of Applied Mathematics and Theoretical Physics,Wilberforce Road, Cambridge, CB3 0WA, UK

Corresponding author email: J.Gjorgjieva@damtp.cam.ac.uk

Keywords: STDP, BCM, triplet model, higher-order correlations, input selectivity

Acknowledgements: We would like to thank Stephen Eglen and Adrienne Fairhall for helpfuldiscussions and reading drafts of this manuscript. This work was funded by a CambridgeOverseas Research Studentship and Trinity College Internal Graduate Studentship (JG), theAgence Nationale de la Recherche grant ANR-08-SYSC-005 (CC), the Wellcome Trust andthe Swiss National Science Foundation (JPP).

Abstract

Synaptic strength depresses for low, and potentiates for high activation of the postsynaptic neuron.This feature is a key property of the well-known Bienenstock-Cooper-Munro (BCM) synaptic rule,which has been shown to maximize the selectivity of the postynaptic neuron, and thereby offersa possible explanation for experience-dependent cortical plasticity such as orientation selectivity.However, this BCM framework is rate-based and a significant amount of recent work has shown thatsynaptic plasticity depends on the precise timing of presynaptic and postsynaptic spikes. In thispaper we consider a triplet model of Spike-Timing Dependent Plasticity (STDP) which dependson the interactions of three precisely-timed spikes. Triplet STDP has been shown to describeexperimentally-observed plasticity which the classical STDP rule, based on pairs of spikes, has failedto capture. In the case of rate-based patterns, we show a tight correspondence between the tripletSTDP rule and the BCM rule. We demonstrate the selectivity property of the triplet STDP ruleanalytically for orthogonal inputs and numerically for non-orthogonal inputs. Moreover, in contrastto BCM, we show that triplet STDP can also elicit selectivity to input patterns consisting of higher-order spatio-temporal correlations. This property is crucial given the large amount of higher-ordercorrelations measured in the brain. We studied the triplet STDP model in a biologically-inspiredsetting from receptive field theory, where orientation and directions selectivity arise due to thesensitivity of triplet STDP to spatio-temporal correlations.

Introduction

Synaptic plasticity depends on the activity of presynaptic and postsynaptic neurons and is believedto provide the basis for learning and memory (1, 2). It has been shown that low frequency stim-ulation (1–3 Hz) (3) or stimulation paired with low postsynaptic membrane potential (4) inducessynaptic long-term depression (LTD), whereas synapses undergo long-term potentiation (LTP) af-ter high frequency stimulation (100 Hz) (5). Such findings are consistent with the well-knownBienenstock-Cooper-Munro (BCM) learning rule (6). This BCM model has been shown to elicitorientation selectivity and other aspects of experience-dependent cortical plasticity (6, 7). Further-more, in this model the modification of the threshold between LTP and LTD varies as a functionof the history of postsynaptic activity, a prediction which has been confirmed experimentally (8).

Despite its consistency with experimental data and its functional relevance, the BCM frameworkis limited in several aspects. Experimentally, since the learning rule is expressed in terms of firingrates, it cannot predict synaptic modification based on the timing of pre- and postsynaptic spikeswhich has attracted particular interest more recently (9, 10). This form of plasticity, called Spike-Timing Dependent Plasticity (STDP) uses the timing of spike pairs to induce synaptic modification(11, 12). Synapses undergo LTP if a presynaptic spike precedes a postsynaptic spike, whereas thereverse timing of pre- and postsynaptic spikes leads to LTD (9, 10, 13, 14). Functionally, the BCMmodel cannot segregate input patterns that are characterized by their spiking temporal structure.

Here, we consider a spike-based learning rule, “the triplet STDP model” (15, 16), and show that

1

it overcomes those two important limitations of the BCM rule, and as a result generalizes the BCMframework. This triplet model uses sets of three spikes (triplets) – instead of pairs of spikes as inthe case of classical STDP – to induce potentiation. More precisely, LTP depends on the intervalbetween the pre- and postsynaptic spikes, and on the timing of the previous postsynaptic spike(Fig. 1A). Furthermore, this triplet learning rule has been shown to explain a variety of synapticplasticity data (17, 18) significantly better than pair-based STDP (16) (see also Fig. 1B).

Computationally, it has been shown that under some rather crude assumptions – when theinput and output neurons have independent Poisson statistics – this triplet model can be mappedto the BCM learning rule (16). In this paper, we take a more biologically-plausible approach byincorporating contributions from input-output spiking correlations in inducing synaptic plasticity.Consistent with results from BCM theory, we demonstrate that in the presence of N orthogonalrate-based patterns, the maximally selective fixed points of the weight dynamics induced by thetriplet rule are stable. Furthermore, we show that the triplet rule acts as a generalized BCM rule inthe sense that postsynaptic neurons become not only selective to rate-based patterns of the inputs,but also to patterns that can be differentiated only based on their spiking correlation structure.

In contrast to existing biophysical learning rules (19) that are consistent with the BCM model,the mathematical simplicity of the triplet model allowed us to characterize the explicit dependenceof the synaptic dynamics on higher-order input correlations. This is of critical importance giventhe ubiquity of those higher-order correlations (20, 21) and their relevance for neural coding (22).

Materials and Methods

Neuronal dynamics. We consider a feedforward network withN input neurons x(t) = [x1(t), . . . , xN (t)]T

where xj(t) =∑

tprejδ(t− tprej ) denotes the Dirac delta spike train at time t of the jth neuron and

tprej its spike times. Input neurons are connected to a single output neuron (Fig. 2A) through aweight vector w(t) = [w1(t), . . . , wN (t)]T giving rise to the postsnynaptic spike train y(t). Theinput spike trains x(t) have instantaneous firing rates ρinst(t) = [ρinst1 (t), . . . , ρ

instN (t)]

T = 〈x(t)〉.(The expectation 〈·〉 is assumed to be taken over the input statistics.) In addition to this firstmoment, the input spike trains have higher-order moments, of which we only considered second(pairwise correlations) and third moments (triplet correlations). The input correlation matrix1 isdefined as C(s) = 1T

∫ T0

〈x(t)xT(t− s)

〉dt where ρ = 1T

∫ T0 ρ

inst(t) dt is the mean firing rate aver-aged over a duration T . Note that this input correlation matrix has an atomic discontinuity ats = 0: C(0) = diag(ρ) δ(0).

The third-order correlation is described by a third-order tensor U whose (k, j, n) componentsare defined as Ukjn(s1, s2) = 1T

∫ T0 〈xk(t)xj(t − s1)xn(t − s2)〉 dt. Like the pairwise correlation

matrix, this third-order tensor also has atomic discontinuities when s1 = 0, s2 = 0 and s1 = s2(see Supplementary Information). For further notational convenience, we let Uj denote a matrixwhose (k, n) element is Ukjn. We assumed that the membrane potential of the postsynaptic neuron

1Formally, the correlation matrix should be written as C(t; s). For notational convenience, we omit here thedependence on t.

2

u(t) increases with the spike times of each input by the excitatory postsynaptic potential (EPSP)scaled by the corresponding weight

u(t) = wT(t)∫ ∞

0�(r)x(t− r) dr [1]

The function �(r) denotes the EPSP kernel with∫∞

0 �(r) dr = 1, taken to be a decaying exponential,1/τm e−r/τm , r ≥ 0, with τm being a membrane time constant of 11 ms (the exact choice of the timeconstant was not important). Postsynaptic spikes were generated stochastically from the membranepotential (23, 24), with a probability density of firing a spike at time t given by the transferfunction 〈y(t)〉 = g(u(t)). For simplicity, we used linear neurons where the transfer function wasapproximated by

g(u(t)) ≈ g(u0) + g′(u0)(u(t)− u0) [2]

where the expected membrane potential averaged over a period T is u0 = 1T∫ T

0 〈u(t)〉 dt = wTρ.

We also used ν = g(u0) to denote the mean postsynaptic firing rate. (The symbol 〈·〉 denotes theexpectation over both the pre- and postsynaptic stochastic spike trains.)

Having described the dynamics of the postsynaptic neuron, we expressed the correlation functionK(s) = 1T

∫ T0 〈y(t)x(t− s)〉 dt between pre- and postsynaptic spike trains solely as a function of the

(convolved) input correlations

K(s) = g′(u0)C�T(s)w + α(u0) ρ [3]

where α(u) = g(u)− g′(u)u, and the superscript � denotes a convolution with respect to the EPSPkernel �(s), i.e. C�(s) =

∫∞0 �(s

′)C(s−s′) ds′ (see Supplementary Information). Note that when thetransfer function is linear g(u) = u, then α(u0) = 0 and K(s) = C�T(s)w. Similarly, by using theapproximation in Eq. 2, the pre-post-post correlation vector Q(s1, s2) = 1T

∫ T0 〈y(t)x(t − s1) y(t −

s2)〉 dt was expressed as

Q(s1, s2) =(g′(u0)

)2∑j

wTU �j (s1, s2)w ej

+ α(u0) g′(u0)(C�T(s1) + C�T(s1 − s2)− 2 ρ ρT

)w + β(u0) ρ [4]

where ej is a vector of zeros with a 1 at the j-th component, and β(u) = g2(u) − g′2(u)u2

(see Supplementary Information). The � superscript in U �j denotes a double convolution, i.e.U �j (s1, s2) =

∫∞0

∫∞0 �(r) �(q)Uj(s1 − r, s2 − r + q) dr dq. When the transfer function is linear

g(u) = u, then α(u0) = β(u0) = 0. In this case, we expressed the pre-post-post correlation vectorQ simply as a function of the (convolved) third-order input correlation tensor U �: Q(s1, s2) =∑

j wTU �j (s1, s2)w ej .

Synaptic dynamics. In the previous section we assumed that the weights w were fixed. Here,we allowed the weights to be dynamic, but with a very small learning rate such that the results inthe previous section were still valid. Consistent with the approach of (23), we expressed the weight

3

change as a Volterra expansion of both the pre- and postsynaptic spike trains. Setting the firstorder terms to zero, we assumed that synaptic plasticity depends on second- and third-order termsonly, i.e. pairs of spikes (1 pre and 1 post) and triplets of spikes (1 pre and 2 post)

ẇ = y(t)∫ ∞

0W2(s)x(t− s) ds+ x(t)

∫ ∞0

W2(−s) y(t− s) ds

+ y(t)∫ ∞

0

∫ ∞0

W3(s1, s2)x(t− s1) y(t− s2) ds1 ds2. [5]

where W2 is the pair-based STDP learning window (11, 23, 25) given by

W2(∆t) =

A+2 e−∆t/τ+ , ∆t ≥ 0−A−2 e∆t/τ− , ∆t < 0 [6]where ∆t = tpost − tpre denotes the timing between a post- and a presynaptic spike, τ+ is thepotentiation time constant and τ− is the depression time constant (Fig. 2B, blue curve). In addi-tion to the spike pair effect described in Eq. 6, spike triplets (tpre, tpost, t′post) also affect synapticpotentiation depending on their timing difference ∆t1 = tpost − tpre and ∆t2 = tpost − t′post

W3(∆t1,∆t2) =

A+3 e−∆t1/τ+e−∆t2/τy ,∆t1 ≥ 0,∆t2 ≥ 00, otherwise. [7]Thus, the level of potentiation induced by pairs of presynaptic and postsynaptic spikes in pair-basedSTDP is modulated by the timing difference between the two postsynaptic spikes, given by the timeconstant τy (Fig. 1A and 2C, left). The parameters used throughout this paper were those of theminimal triplet rule by (26)2, i.e. A+2 = 0, A

−2 = −6.5 × 10−3, A

+3 = 7.1 × 10−3, τ+ = 16.8 ms,

τ− = 33.7 ms, τy = 114 ms. Assuming slow learning dynamics (implying small W2 and W3), wereplaced the weights w by their expectation averaged over a period of T : w(t)← T−1

∫ t+Tt 〈w(s)〉ds

ẇ =∫ ∞−∞

W2(s)K(s) ds+∫ ∞−∞

∫ ∞−∞

W3(s1, s2)Q(s1, s2) ds1 ds2. [8]

The terms arising from the contributions of pairwise and third-order correlations in this equationare illustrated in Fig. 2. By using the expression of the pre-post correlation vector K derived inEq. 3 and the pre-post-post correlation vector Q from Eq. 4 in the case of linear transfer function,we derived the following equation for the weight dynamics

ẇ = Aw +N∑j=1

(wTBjw

)ej [9]

2Note that (26) assumed the wrong number of pairs in the STDP experiments (60 instead of 75) and thereforethe fitted amplitude parameters were overestimated by approximately 10%. For the sake of consistency with (26) wekept the same parameters.

4

where A =∫∞−∞W2(s)C

�T(s) ds is the (convolved) input correlation matrix integrated with thepairwise learning rule W2. Similarly, we have Bj =

∫∞−∞

∫∞−∞W3(s1, s2)U

�j (s1, s2) ds1 ds2 which

corresponds to the integral of the triplet rule W3 with the third-order input statistics. Equivalently,Eq. 9 can be written as

ẇ = φ(ν) ρ+ ∆Aw +N∑j=1

(wT∆Bj w

)ej [10]

where φ(ν) = W 2ν + W 3ν2 corresponds to the BCM term, and W 2 =∫∞−∞W2(s) ds and W 3 =∫∞

−∞∫∞−∞W3(s1, s2) ds1 ds2 are the integrals of the pair-based and triplet STDP rules, respec-

tively. The new covariance terms ∆A and ∆B are defined analogously to A and B: ∆A =∫∞−∞W2(s) ∆C

�T(s) ds and ∆Bj =∫∞

0

∫∞0 W3(s1, s2) ∆U

�j (s1, s2) ds1 ds2. Here, instead of the pair-

wise and triplet input correlations C and U respectively, we subtracted the means to obtain thepairwise covariance matrix ∆C(s) = C(s) − ρ ρT and triplet covariance tensor ∆Ukjn(s1, s2) =Ukjn(s1, s2)− ρk ρj ρn.

Weight dynamics for input selectivity. In the general case, we considered P input patterns,where pattern i has mean firing rate ρ(i), and pairwise and triplet correlation terms A(i) and B(i),respectively. Each input pattern i was associated with a probability pi of occurrence, and gave riseto an average postsynaptic firing rate ν(i) = wTρ(i). The level of selectivity of the postsynapticneuron defined by (6) is given by Sel(w) = 1 −

(∑i piw

T ρ(i))/(maxiwT ρ(i)

). The selectivity is

zero if the output firing rate is identical for each input pattern. The selectivity is maximal is theoutput firing rate is zero in response to all input patterns except to one, when the selectivity isgiven by (N − 1)/N if pi = 1/N .

To match the triplet rule to the BCM model of rate-based selectivity, we set A−2 → A−2 ν̄/ρ

p0,

where the expectation of the pth power of the postsynaptic firing rate can be expressed as ν̄ =∑Pi=1 pi

(ν(i))p

. This quantity was approximated by low-pass filtering the pth power of the instan-taneous postsynaptic firing rate ν(t) = g(u(t)) with a time constant which has to be larger than Ptimes the frequency of pattern presentation, i.e. r ' ν̄ where τrṙ = −r + νp with a time constantof τr = 5 s (unless otherwise noted). For all the calculations in this paper we took p = 2. Usingthe minimal triplet model of (26) where A+2 = 0, A

(i) contains only depression effects from the pairSTDP rule (A−2 6= 0) such that Eq. 10 becomes

ẇ =P∑i=1

pi

φ(ν(i), ν̄) ρ(i) + ∆A(i)(ν̄)w + N∑j=1

(wT∆B(i)j w

)ej

. [11]To find the nontrivial fixed points of this system, we solved ẇ = 0.

Selectivity with N rate-based patterns. Here, we show that the maximally selective fixedpoints are stable points of the triplet STDP rule in the case of independent Poisson inputs. Thefull proof is given in the Supplementary Information. By assuming the minimal triplet model(A+2 = 0) we have ∆A = 0 and ∆B is given by (see Supplementary Information) (∆B

(i)j )kn =(

b1 δkj ρ(i)n + b2 δkn ρ

(i)k + b3 δkj δkn

)ρ

(i)j with b1 =

∫∞0

∫∞0 W3(s1, s2) (�(s1) + �(s2 − s1)) ds1 ds2, b2 =

5

∫∞0

∫∞0 W3(s1, s2)

∫∞0 �(r) �(r− s2) dr ds1 ds2 and b3 =

∫∞0

∫∞0 W3(s1, s2) �(s1) �(s2− s1) ds1 ds2. In

the presence of P Poisson patterns, we can write the expected weight dynamics as

ẇ =P∑i=1

pi

(φ(ν(i), ν̄)1 + Λ(i)

)ρ(i) [12]

where 1 is the identity matrix and Λ(i) is a diagonal matrix where the jth diagonal element isΛ(i)jj =

(b1wj ν

(i) + b2 ν(i)2 + b3w

2j

)ρ

(i)j with ν

(i)2 =

∑k w

2k ρ

(i)k . Solving for the fixed points we find

a total of 2N fixed points of which N are maximally-selective, in agreement with BCM theory (seeSupplementary Information). Moreover, these maximally-selective fixed points are stable, becausethe Jacobian evaluated at each point is a diagonal matrix with negative entries (see SupplementaryInformation). We also calculate the sliding threshold around the maximally-selective fixed points(see the main text).

Correlation-based patterns in a two-dimensional network. Here, we presented patternswith the same rates ρ(1) = ρ(2), but different pairwise and third-order correlations in each pattern.We further imposed a lower bound on the system, w ≥ wmin = 0, which in the case of orthogonalrate-based pattern was automatically satisfied. Then the fixed points of maximum selectivity (w∗1, 0)and (0, w∗2) are given by

w∗1 =

∑i pi

(B

(i)1

)11(∑

i piA(i)11

) Pi pi

“ρ(i)1

”2ρ20

and w∗2 =

∑i pi

(B

(i)2

)22(∑

i piA(i)22

) Pi pi

“ρ(i)2

”2ρ20

[13]

To find the stability of these fixed points, evaluating the Jacobian is not appropriate because ofthe lower bound on the weights. Denoting the system in Eq. 11 by ẇ = F (w) for stability of eachfixed point w∗, two conditions must be satisfied: (1) ∂Fm(w)∂wm

∣∣∣w=w∗

< 0, and (2) Fj 6=m(w∗) < 0.Numerical simulations show that these conditions are always satisfied for the minimal triplet model(see Supplementary Information). This system can be also generalized to two pools of inputsassuming the weights in each pool evolve together, however, extensions to N correlated patternsare currently only possible with numerical simulations.

Simulations of correlated spike trains. Higher-order correlations in the spike trains in Fig. 5were simulated by taking the union of an independent spike train and a common spike train (27–29).This approach was used for the generation of spatial correlations, however, for the spatio-temporalcorrelations we implemented the mixture method described by (29). In short, instantaneous corre-lations were generated by taking the union of an independent spike train and a common spike train(as in the case of spatial correlations), and then the generated spikes were shifted by independentand identically distributed random numbers from an appropriate distribution function. We usedan exponential distribution with a time constant τc (see Supplementary Information). Thus, forsimplicity we studied symmetric correlation functions, and assumed uniform correlations for allinput pairs and triplets.

6

Numerical simulations with multiple patterns. For the simulations with rate-based pat-terns in Fig. 3, the inputs within each pattern were given independent Poisson spike trains lackingcorrelations. In each pattern, 90 inputs had rates of 1 Hz and 10 inputs had rates of 50 Hz. The10 inputs with the high rates were always assigned to a set of neighboring neurons (1–10, 11–20,. . ., 91–100), determining the pattern.

For the simulations with correlation-based patterns in Fig. 5, each of the 100 inputs had thesame firing rate of 10 Hz. In each pattern, 90 inputs were given independent Poisson spikes,and 10 inputs had uniform correlations between any pair and triplet of inputs. For the spatialcorrelations in Fig. 5B, each pair and triplet of inputs shared 90% identical spikes. For the spatio-temporal correlations in Fig. 5C, one half of the 90% shared spikes for each pair and triplet of inputswere shifted by an exponential random distribution with a mean of 5 ms resulting in symmetric,exponentially-decaying correlations with a timescale of 5 ms.

In Fig. 3 and 5, a new randomly-chosen pattern was presented to the network every 200 ms.Pre- and postsynaptic spikes were simulated stochastically given the respective firing rates. Theinitial weights were set to 1 and hard bounds were set between 0 and 3. The target firing rate wasset to ρ0 = 12 Hz. Postsynaptic activity was low-pass filtered with a time constant of 5 seconds. A+2and A−3 were reduced by a factor 10 compared to the parameters in (26) to give smooth evolutionof the weights.

Orientation selectivity and receptive field development. In Fig. 4, every 200 ms (cor-responding to a fixation time between saccades), a small patch of 16 × 16 pixels was randomlychosen from ten natural images. The images were taken from the standard benchmark (30) alreadyprewhitened. In order to remove orientation bias, half of the time the patch was transposed, re-flected about the vertical or the horizontal axis. The inputs of the feedforward network consistedof 16 × 16 independent Poisson processes firing at a rate corresponding to the positive gray levelof the pixel (ON inputs) and another 16× 16 OFF inputs firing proportional to the negative graylevel of the pixel. The initial weights were set to 1 and hard bounds were set between 0 and 2. Thetarget firing rate was set to ρ0 = 5 Hz. A+2 and A

−3 were reduced by a factor 100 compared to the

parameters in (26). Every 50 s, the norm of the ON weights to the one of the OFF weights wasequalized.

For Fig. 7, bars of different orientations: vertical, horizontal, two diagonal directions at ±45◦,which could move in one of the two directions perpendicular to the bar’s orientation were presentedto a feedforward network, giving a total of 8 patterns. In the network, an image of 9× 9 pixels cor-responded to the probability of firing of 81 independent Poisson inputs. When a bar was presented,all the 9 inputs corresponding to the bar fired at 50 Hz. The bar stayed at the given position for10 ms and then shifted to the next pixels in one of the two directions (positive velocity for onedirection and negative for the other). A new randomly-chosen patten was presented to the networkevery 200 ms.

The model for the postsynaptic neuron was the same as described previously, but in additionto an EPSP kernel of 10 ms, an IPSP kernel of 20 ms was also used. This additional IPSP kernel

7

allowed the neuron to be more sensitive to temporal structure of its input (31). The weightsevolved under the minimal triplet rule augmented with a sliding threshold as described above. Alower bound of 0 and an upper bound of 3 was imposed on the weights. Postsynaptic activity waslow-pass filtered with a time constant of 20 seconds.

Results

Triplet STDP implies selectivity with rate-based patterns

Orientation-selective neurons in the primary visual cortex respond strongly when a bar is presentedin a certain orientation and respond much less if the bar is presented in a different orientation (32).This orientation selectivity is learned during receptive field development, and normal patterns ofsensory experience are important for receptive field maturation (33, 34). How is this orientationselectivity, or more generally pattern selectivity learned by a neuronal network? In order to solvethis task, (6) proposed a simple model of synaptic plasticity which is known as the BCM learningrule. In the BCM framework, a randomly chosen input pattern pattern i (of P possible patterns)with rates ρ(i) is presented with probability pi to a feedforward network with N inputs (Fig. 3A).This causes a postsynaptic firing rate ν(i) = wTρ(i), where w is the weight vector. The weightchange induced by the BCM rule is proportional to the input firing rate ρ

ẇ = φ(ν, ν̄) ρ [14]

and scales with a non-linear function φ, which depends not only on the postsynaptic firing rateν, but also on the average (over all patterns) of a non-linear function of the postsynaptic rateν̄ =

∑i pi(ν(i))2

. The non-linear function φ must be negative when the postsynaptic firing rate isbelow a given threshold θ – which itself depends on ν̄ – and positive when it is above (Fig. 3B). Inthe case of orthogonal input patterns, (6) showed that the maximally-selective fixed points of thedynamics (for which the postsynaptic neuron shows the largest response when the selected patternis presented) are stable.

Interestingly, the average weight change under the triplet STDP rule can be written preciselyas the BCM term plus some perturbation terms due to the input correlations

ẇ = φ(ν) ρ+ ∆Aw +N∑j=1

(wT∆Bj w

)ej [15]

where φ(ν) = W 2 ν + W 3 ν2 is the BCM term (Fig. 3B) with W 2 and W 3 being the overall areaunder the pair-based and triplet STDP rules, respectively. The additional terms, ∆A and ∆B,depend on the pairwise and third-order input statistics integrated over the pair-based and tripletSTDP learning rules, respectively (see Materials and Methods). In order to get depression at lowpostsynaptic firing rate and potentiation for higher firing rate, pairs of spikes must have an overalldepressive effect (W 2 < 0) and triplets of spikes must induce potentiation (W 3 > 0). This is indeed

8

the case for the minimal triplet STDP model that has been proposed by (26) and that we considerhere.

There are two differences between the original BCM term in Eq. 14 and the term in the tripletmodel in Eq. 15. Firstly, in the triplet model, the function φ depends only on the temporally-averaged postsynaptic activity ν, whereas in the BCM model, φ also depends on the postsynapticactivity averaged over all patterns, ν̄. However, if we redefine the amplitude parameter for pair-based depression A−2 as A

−2 ν̄/ρ

p0, where ρ0 is a constant denoting the target rate of the postsynaptic

neuron (26), then both φ and ∆A in Eq. 15 will depend on ν̄.The second difference is the presence of the two additional terms (∆A and ∆B) in Eq. 15. If

the inputs are Poisson neurons, we can rewrite Eq. 15 as

ẇ = (φ(ν, ν̄)1 + Λ) ρ [16]

where 1 denotes the identity matrix and Λ is a diagonal matrix (see Materials and Methods). Ifwe now assume that the patterns are orthogonal, we can show that the condition ẇ = 0 in Eq. 16gives rise to 2N fixed points, which is consistent with the results of BCM theory. Moreover, the Nmaximally selective fixed points, w∗(n) = (0, . . . , 0, w∗n, 0, . . . , 0), are stable fixed points (Fig. 3D,Eshows an illustration in two dimensions). The sliding threshold of the system around the fixedpoint (Fig. 3B) is given by

θ(ν̄) = θ0(ν̄)(

1 +(τ1ρ

(n)n

)−1+(τ2ρ

(n)n

)−2)−1[17]

where θ0(ν̄) = A−2 τ−ν̄/(A+3 τ+τyρ

p0) is the sliding threshold of the BCM term only when the correla-

tion terms can be neglected (i.e. ∆A = 0 and ∆B = 0) and τ1 and τ2 are time constants dependingon the parameters of the system (see Supplementary Information).

For non-orthogonal inputs, deriving selectivity analytically is too difficult, so we performednumerical simulations. Ten rate-based non-orthogonal patterns (Fig. 3A) were presented to 100inputs neurons in the feedforward network (Fig. 2A). At the end of the simulation the postsynapticneuron became selective to only one pattern, i.e. only the weights from one set of ten input neurons(61–70) potentiated, while the other inputs depressed (Fig. 3C). The choice of the selected patternis random. Note that standard pair-based STDP also exhibits selectivity of rate-based patterns(12), but the type of competition is of a different nature. As pointed out by (6), the competition inthe BCM rule (and equivalently with triplet STDP) is temporal via the sliding threshold whereaswith a pure Hebbian rule (and equivalently with pair-based STDP), the competition is spatial –between synapses.

Receptive fields development with natural stimuli

We have shown that the triplet STDP rule can be mapped to the BCM model in the case of rate-based patterns presented to a feedforward network. Therefore, we expected that the triplet STDP

9

rule would be able to explain the development of localized receptive fields. In Fig. 4 we providedinput consisting of small patches of natural images chosen from a standard benchmark (30) andwhitened with standard preprocessing (35). The weights in the feedforward network evolved underthe triplet STDP rule (Fig. 4A). After learning, the weights were rearranged on the image, showing astable spatial structure resembling a receptive field (Fig. 4B). Different runs of the same simulationprotocol resulted in receptive fields with different locations and orientations (Fig. 4C).

While classical pair-based STDP has been shown to support selectivity in the case of rate-based patterns (25), the results here suggest that triplet STDP performs a fundamentally differentcomputation than pair-based STDP. The receptive fields shown in Fig. 4 were spatially-localized,which has been previously related to independent component analysis (ICA) (35) and sparse coding(7, 30, 36). This stands in stark contrast to principal component analysis (PCA) of image patches,for example, as implemented with linear neurons and a linear Hebbian rate-based learning rule(37) or with pair-based STDP (data not shown). Unlike most other ICA algorithms (35), however,triplet STDP is supported by experimental plasticity data.

Triplet STDP elicits selectivity with correlation-based patterns

So far we have shown that the triplet STDP model can be mapped to the BCM model of rate-based selectivity for independent patterns based on different firing rates, and can perform ICA-likecomputations. While this is useful because it extends the rate-based framework of the BCM modelto a spike-based rule, the triplet learning rule can be seen as a further generalization of the BCMmodel in the following sense: in Eq. 15, ∆A and ∆B depend on the second and third-order inputcorrelations, respectively, therefore, triplet STDP must be sensitive to spatio-temporal correlationsin the inputs.

In order to examine our hypothesis, we presented 10 ‘correlation-based’ patterns to the feed-forward network in Fig. 2A with 100 inputs. Unlike the rate-based patterns examined so far,correlation-based patterns consisted of the same rates, but had different pairwise and third-ordercorrelations in the inputs (Fig. 5A). Since the firing rates of the inputs in each pattern were thesame, the postsynaptic neuron elicited the same response to each pattern. Therefore, in the case ofcorrelation-based patterns selectivity was defined in terms of the selective potentiation of a group ofinputs which shared nonzero spatio-temporal correlations. Fig. 5B shows a simulation with purelyspatial correlations – the correlation structure between any pair or triplet of inputs was instanta-neous and had no temporal structure. The weights from one pattern potentiated (from a set of teninput neurons with indices 41–50), while the other weights depressed. Increasing the complexityof the correlation structure, in Fig. 5C spatio-temporally correlated inputs were presented to thenetwork. Despite the temporal structure diluting the strength of the input correlations, a set often weights (81–90) characteristic of one pattern potentiated, while the other weights depressed.

We also found that the map between the BCM rule and the triplet STDP model can be es-tablished even in the case of correlation-based patterns. To see this, in Fig. 5D we illustrate theexpected weight change as a function of the postsynaptic firing rate in the feedforward network

10

receiving spatio-temporal correlation-based patterns. While the additional terms ∆A and ∆Bin Eq. 15 prevent us from deriving an explicit expression for the modification threshold, such athreshold still exists and depends on the correlation strength of the inputs. Example curves fortwo different input correlation strengths are presented in Fig. 5D, demonstrating the dependenceof the effective threshold for potentiation versus depression on correlation strength.

Due to the increased complexity of the system when correlations exist in the inputs, we derivedthe fixed points of maximal selectivity (w∗1, 0) and (0, w

∗2) in a small network of two inputs neurons

(or two input pools), and analyzed their stability (see Materials and Methods). The correlationterms ∆A and ∆B introduced additional nonlinearities in the weight dynamics in Eq. 15, such thata lower bound had to be introduced to prevent the weights from becoming negative (in agreementwith Dale’s law). For the two-dimensional network, we found the maximally-selective fixed pointsto always be stable. Fig. 5E shows the two-dimensional phase plane, where the two unstable fixedpoints (red symbols) drive the weight trajectories toward the axes where the stable maximally-selective fixed points are located (black symbols). Example weight trajectories are shown in Fig. 5Ffor one choice of initial condition.

However, for networks with more than two input pools, the correlation structure of the inputscan have a significant effect on the weight dynamics. We extended the simulation in Fig. 5C toexamine how the temporal correlation structure of the inputs influences the selective potentiationof synaptic weights corresponding to different patterns. We studied a particular example of asymmetric spatio-temporal correlation: exponentially-decaying in time, which was the same forall pairs and the same for all triplets of inputs (see Materials and Methods). Therefore, whilepreserving the peak of correlation strength, we examined the role of the correlation structure on theselective potentiation of synaptic weights by studying a single parameter: the correlation timescale(Fig. 5G). Increasing the correlation timescale resulted in a ‘dilution’ of the correlation strength.The triplet STDP rule failed to consistently potentiate the weights of one input pattern and oftentwo or three patterns were simultaneously selected (Fig. 5G). This result suggests that correlationsover broad timescales fail to evoke selective potentiation of correlation-based input patterns, andcould be used to understand the implications of different correlation structure occurring in differentbrain regions.

Triplet STDP, but not pair-based STDP, can elicit selectivity driven by third-

order correlations

We have shown that triplet STDP (because of its link to the BCM model) can do ICA-like com-putations, resulting in the development of spatially-localized receptive fields (Fig. 4). While thiscontrasts classical Hebbian learning rules, including pair-based STDP, we asked whether tripletSTDP can do other computations which pair-based STDP cannot. We have also shown that tripletSTDP can induce selectivity driven by rate-based (Fig. 3) and correlation-based patterns (Fig. 5).Previous work has shown that pair-based STDP can also elicit rate-based input selectivity (25, 38),but it has not been studied how pair-based STDP performs in the case of correlation-based patterns.

11

To answer this question, we designed a simpler selectivity task consisting of two spatial correlation-based patterns presented to a feedforward network of 6 input neurons. The two patterns consistedof the same firing rates, the same pairwise correlations, but differed in the presence or absence ofthird-order correlations (Fig. 6A). The probabilities of presentation of the two patterns were varied,as the weights evolved under triplet STDP (Fig. 6B) and pair-based STDP (Fig. 6C). If p1 denotesthe probability with which the first pattern (with third-order input correlations) was presented tothe neurons in the first group, then the probability that the weights corresponding to this patternwin is shown in Fig. 6B,C. For the static pattern case when p1 = 1 (corresponding to the twopatterns always being presented to the same group of neurons in the network), the triplet STDPrule selectively potentiated the weights of the pattern with third-order correlations. However, thepair-based STDP rule continued selectively potentiating the weights from either inputs group withequal probability.

In summary, when weight dynamics evolve under pair-based STDP, the rule is not sensitive tothe third-order correlations which distinguish one pattern from another, thus, regardless of p1 therule behaves as if two identical patterns were presented to the network. In contrast, when thenweights evolve under triplet STDP, then the rule almost always selects the pattern with the third-order correlation for p1 = 1. This demonstrates that the triplet STDP rule can distinguish betweeninputs solely based on the higher-order correlation structure, which pair-based STDP ignores. Asa result, triplet STDP will be computationally more powerful in systems where such higher-ordercorrelations have been characterized (20–22).

Spatio-temporal receptive field development

Finally, we illustrate the selectivity property of the triplet rule shown analytically, in a biologically-inspired setting from visual cortex receptive field theory. We presented eight different patternsconsisting of four bars of different orientations (horizontal, vertical, and two diagonals tilted at±45◦) each moving in one of the two directions perpendicular to the bar’s orientation, as in Fig. 7A,to a feedforward network. The input neurons in the network were organized in a 9×9 grid mappingto a single postsynaptic neuron as in Fig. 7A. Each input spike produced both an excitatorypostsynaptic potential (EPSP) and an inhibitory postsynaptic potential (IPSP) with a longer timeconstant (see Materials and Methods). This additional IPSP makes the postsynaptic neuron moresensitive to the temporal derivative of the input currents (31). Subjected to learning governed bytriplet STDP, the synapses in the network selectively refined, some depressing to the lower bound ofzero and others potentiating maximally to the upper bound (Fig. 7B). At the end of the simulationthe postsynaptic neuron became selective to one of the bars, as shown in Fig. 7C by the strongweights corresponding to inputs placed in the bar with a vertical orientation.

In addition to having a spatial structure as given by the orientation selectivity, the receptive fieldin this scenario also developed a temporal structure as a result of the spatio-temporal correlationsimposed by the moving bars. In order to visualize this effect, the weights were frozen after learningand the averaged firing rate of the postsynaptic neuron was recorded as all four bars were re-

12

presented to the network, but at different velocities. The postsynaptic neuron fired the most whenthe bar with vertical orientation was presented at the velocity used during the learning (Fig. 7D).Therefore, the postsynaptic neuron became selective to both orientation and velocity.

Recently, after training with moving stimuli, (39) observed the emergence of direction selectivityin cortical neurons of visually naive ferrets for the trained directions of motion. Motivated by thisexperimental result, we plotted the time evolution of the averaged firing rate for bars of all fourorientations moving in each direction in the scenario described above. We found that in addition toorientation selectivity, the triplet STDP rule also induced direction selectivity in the postsynapticneuron. Thus, while the postsynaptic neuron had the highest firing rate for the bar in the verticalorientation relative to all other orientations, it had a higher firing rate for the bar moving to theleft (∼ 40 Hz), compared to the firing rate for the bar moving to the right (∼ 30 Hz) at the end ofthe simulation.

In this scenario the development of direction selectivity relied on two elements. First, tripletSTDP was sensitive to the spatio-temporal input correlations, and second, we used a modifiedpostsynaptic potential kernel which made the postsynaptic neuron sensitive to temporal derivativesof the input current. This last requirement was essential for obtaining direction selectivity becauseeach pattern (bar with a given orientation and direction) activated every input the same numberof times during its presentation. If the postsynaptic potential was only excitatory, then the totalinput to the postsynaptic neuron would be the same regardless of the direction of the stimulation.More robust direction selectivity as observed biologically (32, 40) can most likely be obtained byusing a more elaborate neuron that is strongly dependent on the temporal order, e.g. a neuron withfiring rate adapting properties, or a plastic recurrent network with unidirectional connections alongthe selected direction.

Discussion

The biologically-plausible BCM theory is attractive because it exhibits selectivity (6) and performsICA computation (41). Synaptic plasticity, however, is shown to depend on the precise spike timing(9, 10) classically modeled by pair-based STDP (11, 12). In this paper, we claim that a differentspike-based rule, triplet STDP, which is known to describe plasticity experiments accurately (26),exhibits the computational properties of the BCM rule, and also acts as a generalized BCM rulebecause of its sensitivity to spatio-temporal correlations.

Rate-based patterns. First, we showed that triplet STDP supports pattern selectivity whenthe patterns are determined by firing rates. Consistent with the BCM theory, the maximally-selective fixed points of the weight dynamics under triplet STDP are always stable. Triplet STDPachieved rate-based selectivity even though the equations for the weight dynamics under tripletSTDP contained contributions from the input auto-correlations. In addition, triplet STDP performsICA computation allowing the development of localized receptive fields. This would not be possiblewith pair-based STDP which only performs PCA.

13

Correlation-based patterns. Second, the spiking-based implementation of triplet STDP allowedus to extend the analysis beyond BCM to correlation-based patterns, determined purely by thespatio-temporal input correlations. In this case, we showed that triplet STDP can select for patternswhich consist of higher-order correlations, when classical pair-based STDP fails. The contributionof spike timing correlations has recently been extensively studied in different circuits (20–22, 42).However, even though higher-order statistical dependencies have been observed in many brain areas,their role in the processing of sensory information has been debated (43, 44). Due to its sensitivityto higher-order correlations widely observed in the nervous system, in contrast to earliest modelsof synaptic plasticity (rate-based (6, 45, 46) and pair-based STDP (11, 12)), triplet STDP offers anadditional computational advantage over other learning rules. Although when studying selectivitydriven by input patterns determined purely by the input correlations we could not use the definitionof selectivity as in the BCM theory, we still observed the selective potentiation of synaptic weightscorresponding to input patterns with nonzero correlations. We observed that increasing the inputcorrelation timescale dilutes the correlation strength which prevents the cooperation of inputsnecessary for the emergence of selectivity. Therefore, our results make experimental predictionsabout the type of correlation structure which leads to selectivity.

Other Models. We showed that triplet STDP is a biologically-plausible learning rule which canperform ICA-like computations (35, 41). Recently, (47) showed that a voltage-based learning ruleconsistent with triplet STDP also supports the development of localized receptive fields. However,the rule was too complex to analyze mathematically. Several other models have addressed the issueof spiking-based implementations of the BCM learning framework. Ref. (48) and (49) established acorrespondence between classical pair-based STDP and BCM under certain assumptions, however,they failed to demonstrate an exact mapping and to derive a modification threshold. Those twomodels, as well as the classical pair-based STDP, are inconsistent with a variety of experimentaldata such as frequency dependence (17, 50), in contrast to triplet STDP. Ref. (51) derived analternate spike-based learning rule designed to maximize the information transmission between anensemble of inputs and the output of a postsynaptic neuron. Though such plasticity rules derivedfrom the infomax principle can generalize the BCM theory to spiking neurons (52, 53) and canbe reduced under some assmptions to the triplet STDP rule (54), the dynamics of these rules arerather complicated to be studied analytically in contrast to the triplet STDP model. The sameproblem arises with a different class of biophysical models (19, 55, 56), which have primarily beenstudied numerically. Therefore, the triplet STDP model is a good trade off: it can reproduce alarge set of electrophysiological data, and yet has a relatively simple formulation allowing analyticaltreatment.

Extensions. The analytical study presented in this paper uses a simple neuron model. In particu-lar, there is no threshold membrane potential for the occurrence of spikes, nor a spike afterpotential.Several extensions can be performed which include a more realistic non-linear transfer function forthe neuron, inhibitory inputs, the presence of a refractory period or firing rate adaptation. Fur-thermore, the current results were derived for a feedforward network with a single postsynaptic

14

neuron. Analytical calculations of recurrently-connected networks with plastic weights are alreadysophisticated for pair-based STDP (57–61), but with a theory for the simple feedforward networkin place, those calculations could be addressed in the future for the triplet model as well.

15

References

[1] Bliss, TV, Collingridge, GL (1993) A synaptic model of memory: long-term potentiation inthe hippocampus. Nature 361:31—39.

[2] Malenka, RC, Nicoll, RA (2009) Long-Term Potentiation–a decade of progress? Science285:1870–1874.

[3] Dudek, SM, Bear, MF (1992) Homosynaptic long-term depression in area CA1 of hippocampusand effects of N-methyl-D-aspartate receptor blockade. Proc Natl Acad Sci USA 89:4363–4367.

[4] Artola, A, Bröcher, S, Singer, W (1990) Different voltage-dependent thresholds for inducinglong-term depression and long-term potentiation in slices of rat visual cortex. Nature 347:69–72.

[5] Bliss, TV, Lomø, T (1973) Long-lasting potentiation of synaptic transmission in the dentatearea of the anaesthetized rabbit following stimulation of the perforant path. J Physiol (Lond)232:331–356.

[6] Bienenstock, EL, Cooper, LN, Munro, PW (1982) Theory of the development of neuronselectivity: orientation specificity and binocular interaction in visual cortex. J Neurosci 2:32–48.

[7] Cooper, LN, Intrator, N, Blais, BS, Shouval, HZ (2004) Theory of cortical plasticity (WorldScientific, Singapore).

[8] Kirkwood, A, Rioult, MC, Bear, MF (1996) Experience-dependent modification of synapticplasticity in visual cortex. Nature 381:526–528.

[9] Markram, H, Lübke, J, Frotscher, M, Sakmann, B (1997) Regulation of synaptic efficacy bycoincidence of postysnaptic AP and EPSP. Science 275:213–215.

[10] Bi, GQ, Poo, MM (1998) Synaptic modifications in cultured hippocampal neurons: dependenceon spike timing, synaptic strength, and postsynaptic cell type. J Neurosci 18:10464–10472.

[11] Gerstner, W, Kempter, R, van Hemmen, JL, Wagner, H (1996) A neuronal learning rule forsub-millisecond temporal coding. Nature 383:76–78.

[12] Abbott, LF, Nelson, SB (2000) Synaptic plastictiy - taming the beast. Nat Neurosci 3:1178–1183.

[13] Zhang, LI, Tao, HW, Holt, CE, Harris, WA, Poo, MM (1998) A critical window for cooperationand competition among developing retinotectal synapses. Nature 395:37–44.

[14] Feldman, DE (2000) Timing-based LTP and LTD at vertical inputs to layer II/III pyramidalcells in rat barrel cortex. Neuron 27:45–56.

16

[15] Pfister, JP, Toyoizumi, T, Barber, D, Gerstner, W (2006) Optimal spike-timing-dependentplasticity for precise action potential firing in supervised learning. Neural Comp 18:1318–1348.

[16] Pfister, JP, Gerstner, W (2006) in Adv Neur Inf Proc Sys 18, eds Weiss, Y, Schölkopf, B,Platt, J (MIT Press, Cambridge, MA), pp 1083–1090.

[17] Sjöström, PJ, Turrigiano, GG, Nelson, SB (2001) Rate, timing, and cooperativity jointlydetermine cortical synaptic plasticity. Neuron 32:1149–1164.

[18] Wang, HX, Gerkin, RC, Nauen, DW, Wang, GQ (2005) Coactivation and timing-dependentintegration of synaptic potentiation and depression. Nat Neurosci 8:187–193.

[19] Shouval, HZ, Bear, MF, Cooper, LN (2002) A unified model of NMDA receptor-dependentbidirectional synaptic plasticity. Prov Natl Acad Sci USA 99:10831–10836.

[20] Montani, F et al. (2009) The impact of high-order interactions on the rate of synchronousdischarge and information transmission in somatosensory cortex. Phil Trans R Soc A 367:3297–3310.

[21] Ohiorhenuan, IE et al. (2010) Sparse coding and high-order correlations in fine-scale corticalnetworks. Nature 466:617–621.

[22] Pillow, JW et al. (2008) Spatio-temporal correlations and visual signalling in a completeneuronal population. Nature 454:995–999.

[23] Kempter, R, Gerstner, W, van Hemmen, JL (1999) Hebbian learning and spiking neurons.Phys Rev E 59:4498–4514.

[24] Gerstner, W, Kistler, WK (2002) Spiking Neuron Models (Cambridge University Press, Cam-bridge UK).

[25] Song, S, Miller, KD, Abbott, LF (2000) Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nat Neurosci 3:919–926.

[26] Pfister, JP, Gerstner, W (2006) Triplets of spikes in a model of spike timing-dependent plas-ticity. J Neurosci 26:9673–9682.

[27] Gütig, R, Aharonov, R, Rotter, S, Sompolinsky, H (2003) Learning input correlations throughnon-linear temporally asymmetric Hebbian plasticity. J Neurosci 23:3697–3714.

[28] Kuhn, A, Aertsen, A, Rotter, S (2003) Higher-order statistics of input ensembles and theresponse of simple model neurons. Neural Comp 15:67–101.

[29] Brette, R (2009) Generation of correlated spike trains. Neural Comp 21:188–215.

[30] Olshausen, BA, Field, DJ (1996) Emergence of simple-cell receptive field properties by learninga sparse code for natural images. Nature 381:607–609.

17

[31] Rao, RP, Sejnowski, TJ (2003) Self-organizing neural systems based on predictive learning.Philos Transact A Math Phys Eng Sci 361:1149–75.

[32] Hubel, DH, Wiesel, TN (1962) Receptive fields, binocular interaction and functional architec-ture in the cat’s visual cortex. J Physiol 160:106–154.

[33] White, LE, Coppola, DM, Fitzpatrick, D (2001) The contribution of sensory experience to thematuration of orientation selectivity in ferret visual cortex. Nature 411:1049–1052.

[34] White, LE, Fitzpatrick, D (2007) Vision and cortical map development. Neuron 56:327–338.

[35] Hyvärinen, A, Karhunen, J, Oja, E (2001) Independent Component Analysis (Wiley, NewYork).

[36] Blais, BS, Intrator, N, Shouval, H, Cooper, L (1998) Receptive field formation in natural sceneenvironments. comparison of single-cell learning rules. Neural Comp 10:1797–1813.

[37] Oja, E (1982) A simplified neuron as a principal component analyzer. J Math Biol 15:267–273.

[38] Song, S, Abbott, LF (2001) Cortical development and remapping through spike timing-dependent plasticity. Neuron 32:339–350.

[39] Li, Y, Hooser, SDV, Mazurek, M, White, LE, Fitzpatrick, D (2008) Experience with movingvisual stimuli drives the early development of cortical direction selectivity. Nature 456:952–956.

[40] De Valois, RL, Yund, EW, Hepler, N (1982) The orientation and direction selectivity of cellsin macaque visual cortex. Vision Research 22:531–544.

[41] Intrator, N, Cooper, LN (1992) Objective function formulation of the BCM theory of visualcortical plasticity: Statistical connections, stability conditions. Neural Networks 5:3–17.

[42] Luczak, A, P. Barthó, SLM, Buzsáki, G, Harris, KD (2007) Sequential structure of neocorticalspontaneous activity in vivo. Proc Natl Acad Sci USA 104:347–352.

[43] Shadlen, M, Newsome, W (1998) The variable discharge of cortical neurons: implications forconnectivity, computation and information coding. J Neurosci 18:3870–3896.

[44] Latham, PE, Nirenberg, S (2005) Synergy, redundancy and independence in population codes,revisited. J Neurosci 25:5195–5206.

[45] Sejnowski, TJ (1977) Storing covariance with nonlinearly interacting neurons. J Math Biol4:303–321.

[46] Miller, KD (1994) A model for the development of simple cell receptive fields and the orderedarrangement of orientation columns through activity dependent competition between ON- andOFF-center inputs. J Neurosci 14:409–441.

18

[47] Clopath, C, Büsing, L, Vasilaki, E, Gerstner, W (2010) Connectivity reflects coding: a modelof voltage-based STDP with homeostasis. Nat Neurosci 13:344–352.

[48] Izhikevich, EM, Desai, NS (2003) Relating STDP to BCM. Neural Comp 15:1511–1523.

[49] Senn, W, Tsodyks, M, Markram, H (2001) An algorithm for modifying neurotransmitterrelease probability based on pre- and postsynaptic spike timing. Neural Comp 13:35–67.

[50] Clopath, C, Gerstner, W (2010) Voltage and Spike Timing Interact in STDP - A UnifiedModel. Front Synaptic Neurosci 2:25.

[51] Toyoizumi, T, Pfister, JP, Aihara, K, Gerstner, W (2005) Generalized Bienenstock-Cooper-Munro rule for spiking neurons that maximizes information transmission. Proc Natl Acad SciUSA 102:5239–5244.

[52] Toyoizumi, T, Pfister, JP, Aihara, K, Gerstner, W (2005) in Adv Neur Inf Proc Sys 17, edsSaul, L, Weiss, Y, Bottou, L (MIT Press, Cambridge, MA), pp 1409–1416.

[53] Toyoizumi, T, Pfister, JP, Aihara, K, Gerstner, W (2007) Optimality model of unsupervisedspike-timing dependent plasticity: Synaptic memory and weight distribution. Neural Comp19:639–671.

[54] Hennequin, G, Gerstner, W, Pfister, JP (2010) STDP in adaptive neurons gives close-to-optimal information transmission. Front Comput Neurosci 4:1–16.

[55] Castellani, GC, Quinlan, EM, Cooper, LN, Shouval, HZ (2001) A biophysical model of bidi-rectional synaptic plasticity: dependence on AMPA and NMDA receptors. Proc Natl Acad SciUSA 98:12772–12777.

[56] Shouval, HZ, Castellani, GC, Blais, BS, Yeung, LC, Cooper, LN (2002) Converging evidencefor a simplified biophysical model of synaptic plasticity. Biol Cybern 87:383–391.

[57] Burkitt, AN, Gilson, M, van Hemmen, JL (2007) Spike-timing-dependent plasticity for neuronswith recurrent connections. Biol Cybern 96:533–546.

[58] Gilson, M, Burkitt, AN, Grayden, DB, Thomas, DA, van Hemmen, JL (2009) Emergence ofnetwork structure due to spike-timing-dependent plasticity in recurrent neuronal networks. I.Input selectivity–strengthening correlated input pathways. Biol Cybern 101:81–102.

[59] Gilson, M, Burkitt, AN, Grayden, DB, Thomas, DA, van Hemmen, JL (2009) Emergence ofnetwork structure due to spike-timing-dependent plasticity in recurrent neuronal networks. II.Input selectivity–symmetry breaking. Biol Cybern 101:103–114.

[60] Gilson, M, Burkitt, AN, Grayden, DB, Thomas, DA, van Hemmen, JL (2009) Emergence ofnetwork structure due to spike-timing-dependent plasticity in recurrent neuronal networks III:Partially connected neurons driven by spontaneous activity. Biol Cybern 101:411–426.

19

[61] Pfister, JP, Tass, PA (2010) STDP in oscillatory recurrent networks: Theoretical conditionsfor desynchronization and applications to deep-brain stimulation. Frontiers Comp Neurosci4:1–10.

20

B

−20 −10 0 10 20−50

0

50

100

150

200

[ms]

[%]

0.1Hz

20Hz

50Hz

A pre

post

−50 0 50

-100

-50

0

50

100

[ms]

=10 ms

=50 ms

=100 ms

[%]

Figure 1: The triplet STDP rule. A. Synaptic depression is induced as in classical pair-basedSTDP using spike pairs separated by ∆t1 = tpost− tpre < 0. Synaptic potentiation is induced usingtriplets of spikes consisting of two postsynaptic spikes and one presynaptic spike based on timinginterval between them ∆t1 = tpost − tpre > 0 and ∆t2 = tpost − t′post > 0. (See also Fig. 2B for athree-dimenstional representation of the rule.) B. Synaptic change as a function of the time betweenpre- and postsynaptic spikes in a protocol where 60 pairs were presented at different frequencies0.1, 20, and 50 Hz. Depression predominated at low frequency, whereas potentiation was moreprevalent at high frequencies. The data points are experiments performed by (17) and the lines aregenerated with the triplet STDP rule with the parameters taken from (26).

B

-50 0 50

0

50

0 500

1

2

x 105

0

3

6

x 10-3

-50 0 50

0

50

0 50

0

50

0 50

C

−6

−3

0

3

6x 10

−3

−6

−3

0

3

6x 10

−3

0

1000

2000

0

3

6

x 10-3

0

50

0 50

E

0

1000

2000D

. . .

A

0

1

2

x 105

Figure 2: Weight dynamics depend on pairwise and triplet input correlations. A. Themodelling framework consists of a feedforward network of N input spiking neurons connectedthrough the weight vector w = [w1, . . . , wN ]T to a single postsynaptic neuron. B and C. Theweight dynamics in the case of independent Poisson inputs. B. The pairwise contribution consistsof the integral of the pair-based learning window W2 (blue line) and the pre-post correlation vectorKj (red analytics, black numerics). C. The triplet contribution is obtained by multiplying thetriplet learning window W3 (left) with the pre-post-post correlation vector Qj (right). D and E .Same as in B and C, but with correlated inputs.

0 2 4 6

0

2

4

6

0 5 100

2

4

6

C

time [a.u.]

E

time [hr]

inp

ut n

um

be

r

100

80

60

40

20

5 10 15 0

1

2

3

D

0 50 100

0

1

[Hz]

θ = 90 Hz

θ = 65 Hz

θ = 28 Hz

B

...

pattern 10pattern 1 pattern 2 pattern 3

input number

input

firing rate

A

Figure 3: Triplet STDP implies selectivity with rate-based patterns. A. Ten rate-basedpatterns which differ by their firing rates were presented to a feedforward network. B. ∆w as afunction of postsynaptic activity for three different values of the threshold θ between depressionand potentiation. The inputs were independent Poisson spike trains. Symbols denote numerics andlines analytics. C. Evolution of the weights illustrates selectivity in the case of rate-based patterns.Ten rate-based patterns (no correlations) were presented to a feedforward network as in Fig. 1Awith 100 inputs: 90 inputs with rates 1 Hz and 10 inputs with rates 50 Hz. The 10 inputs withhigh rate in each pattern were neighboring (1–10, 11–20, . . . 91–100). A new randomly-chosenpattern was presented to the network every 200 ms. D. Two-dimensional phase plane analysis forrate-based patterns. Nullclines in green and purple intersect at the equilibria shown in red. E. Anexample trajectory for the two weights attracted to one of the stable nodes in D. (a.u. arbitraryunits)

filterOFF weightsON weights

- =

C

A

time [hr]in

pu

t n

um

be

r

0 2 4

100

200

300

400

500

B

Figure 4: Receptive fields development with natural images. A small patch was chosenrandomly from the whitened natural images benchmark (30) every 200 ms. The positive level ofgray of the 16× 16 patch pixels corresponded to the firing rate of the 16× 16 Poisson ON inputs,the negative level to the 16×16 OFF inputs. A. The weights of the ON inputs and the OFF inputsrearranged on a 16× 16 grid after learning. The filter, calculated by subtracting the OFF weightsfrom the ON weights, represents an oriented localized receptive field. B. Temporal evolution of theweights. C. Examples for ten different neurons.

0 1 20

0.5

1

1.5

2

0 5 100

0.5

1

1.5

2

B

FEtime [hr]

5 10 15 0

1

2

3

time [hr]

input

num

ber

10080604020

5 10 15

time [a.u.]

C

...

pattern 10pattern 1 pattern 2 pattern 3

input number

input firing rate

correlationstrength

A

0 50 100

0

2

4D θ = 45 Hzθ = 70 Hz

[Hz]G

% ca

ses o

ne p

atte

rn

is se

lecte

d

correlation timescale [ms]2 4 6 8 10 150

20

40

60

80

100

Figure 5: Triplet STDP implies selectivity with correlation-based patterns. A. Tencorrelation-based patterns which have the same firing rates, but different correlation strength. B.Evolution of the weights illustrates selectivity in the case of correlation-based patterns. Simulationscenario as in Figure 3B, except that the rate of each of the 100 inputs was set to 10 Hz: 90inputs had no correlations (independent Poisson spike trains) and 10 inputs had strong purelyspatial correlations (90% identical spikes). C. Same as B except for the 10 correlated inputs wherespatio-temporal input correlations with a time constant of 5 ms were used. D. ∆w as a function ofpostsynaptic activity for two different input correlation strengths, changing the effective thresholdfor potentiation. Symbols denote numerics and lines analytics. E. Two-dimensional phase planeanalysis for the spatial correlation-based patterns. Nullclines in green and purple intersect at theunstable fixed points shown in red. Imposing a lower bound at 0 resulted in stable maximally-selective fixed points on the axes shown in black. F. An example trajectory for the two weightsattracted to one of the black equilibria in D. G. Percent cases (out of 100 trials) where all inputsfrom a single pattern selectively potentiated. Correlation-based patterns were used with symmetric-exponentially decaying correlations (inset) whose correlation timescale progressively increased.

1.0

0.8

0.6

0.4

0.2

00.6 0.80.70.5 0.9 1.0 0.6 0.80.70.5 0.9 1.0

B

pro

ba

bili

ty o

f g

rou

p 1

win

nin

g

Ctriplet STDP pair STDP

pattern 1 pattern 2

A

input

firing rate

third-order

correlation

strength

pairwise

Figure 6: Triplet STDP, and not pair-based STDP, can distinguish between patternsdetermined by third-order correlations. A. Two groups of inputs each consisting of threeneurons were presented, which had the same rates, and the same pairwise spatial correlations. Theneurons in group 1 had spatial third-order correlations as strong the pairwise correlations, whilethe neurons in group 2 lacked any third-order correlations. The probability of presenting pattern 1to group 1 was denoted by p1. B. Input selectivity under triplet STDP. C. Input selectivity underpair-based STDP. Mean ± s.e.m. were computed from 200 simulations in each case.

time [min]

inp

ut n

um

be

r

0 10 20 30

20

40

60

80

0

1

2

3

0

1

2

3

time [min]

0 10 20 3010

20

30

[Hz]

A B C

D

x

y

x

y

E

−20 0 2010

20

30

40

pres time [ms]

firin

g r

ate

[H

z]

Figure 7: Triplet STDP leads to spatio-temporal receptive field development. A. Fourdifferent bars (horizontal, vertical, and the two diagonals on a 9 × 9 pixels image) were presentedas inputs to a feedforward network with a single postsynaptic neuron, each bar can moving in oneof two directions, giving a total of eight patterns. B. Time evolution of the 81 synaptic weights.C. Final weights re-ordered in a grid corresponding to the input location. D. After learning, theweights were frozen during a testing phase. The four moving bars were presented again to thenetwork with different velocities (positive and negative giving the two directions of motion) whilethe firing rate was measured (averaged over 500 seconds). The training velocity corresponded to apresentation time of −10 ms. E. Time evolution of the firing rate of the postsynaptic neuron forthe eight different patterns. Time axes was cut in bins of 200 seconds to compute a robust averageof the firing rate.

Supplementary Text 1

Supplementary text for the manuscript “ A triplet model of spike-timing dependent plas-ticity generalizes the BCM rule to spatio-temporal inputs” by J. Gjorgjieva, C. Clopath,J. Audet and J.-P. Pfister

Derivation of the correlation functions

We derive the pre-post correlation vector K from Eqn. 5 using pair-based STDP. Given the meanpostsynaptic firing rate ν = g(u0) and using the superscript � to denote a convolution with theEPSP kernel �(s) (see the main text), the pre-post correlation vector K in index notation is

Kj(s) =1T

∫ T0〈y(t)xj(t− s)〉 dt (S1)

=1T

∫ T0〈g(u(t))xj(t− s)〉dt (S2)

=1T

∫ T0

(g(u0) ρj(t− s) + g′(u0) 〈u(t)xj(t− s)〉 − g′(u0)u0 ρj(t− s)

)dt (S3)

= g(u0) ρj + g′(u0)(

1T

∫ T0〈u(t)xj(t− s)〉 dt− u0 ρj

)(S4)

=(g(u0)− g′(u0)u0

)ρj + g′(u0)

∑k

wkC�kj(s) (S5)

= g′(u0)∑k

wk C�kj(s) + α(u0) ρj (S6)

where α(u) = g(u)− g′(u)u. This is Eqn. 3 in index notation.For the post-pre-post correlation tensor Q, we ignore the case when the two postynaptic

spikes overlap (s2 = 0) because it is accounted in the contribution from the pair rule. Then Qof Eqn. 4 in index notation becomes

Qj(s1, s2) =1T

∫ T0〈y(t− s2)xj(t− s1) y(t)〉 dt (S7)

=1T

∫ T0〈g(u(t− s2))xj(t− s1) g(u(t))〉 dt (S8)

= g2(u0) ρj − (g′(u0))2 u20 ρj − 2g′(u0)(g(u0)− g′(u0)u0

)u0 ρj

+ g′(u0)(g(u0)− g′(u0)u0

) 1T

∫ T0

(〈xj(t− s1)u(t)〉+ 〈u(t− s2)xj(t− s1)〉) dt

+ (g′(u0))21T

∫ T0〈u(t)u(t− s2)xj(t− s1)〉 dt (S9)

Supplementary Text 2

Using the expression for α above and letting β(u) = g′(u) (g(u)− g′(u)u) we get

Qj(s1, s2) = β(u0) ρj − α(u0) g′(u0) (2u0 ρj)

+ α(u0) g′(u0)(

1T

∫ T0〈xj(t− s1)u(t)〉 dt+

1T

∫ T0〈u(t− s2)xj(t− s1)〉 dt

)+ (g′(u0))2

(1T

∫ T0〈u(t)u(t− s2)xj(t− s1)〉 dt

). (S10)

Now using the expression for u0 =∑

j wj ρj , we get

Qj(s1, s2) = β(u0) ρj + α(u0) g′(u0)∑k

wk(C�kj(s1) + C

�kj(s1 − s2)− 2u0 ρj

)+ (g′(u0))2

∑k,n

wk wn U�kjn(s1, s2). (S11)

where U �j (s1, s2) =∫∞0

∫∞0 �(r) �(q)Uj(s1 − r, s2 − r + q) dr dq. Eqn. S11 corresponds to Eqn. 4

in the main text but in index notation. Note that this third-order correlation function U hasatomic discontinuities, and we can write

Ukjn(s1, s2) = U◦kjn(s1, s2) + δkj δjn δ(s2 − s1) δ(s1) ρk(t)+ δkn δ(s2)Ckj(s1) + δjn δ(s2 − s1)Ckj(s1) + δkj δ(s1)Ckn(s2) (S12)

where U◦kjn(s1, s2) is the third-order correlation without atomic discontinuities and is equal toρkρjρn if the inputs are Poisson.

Selectivity with N orthogonal rate-based input patterns

We show that the maximally selective fixed points are stable points of the triplet STDP rule inthe case of independent Poisson inputs. We use the minimal triplet model (A+2 = 0) such that∆A = 0. By using Eqn. S11 and S12, (∆B(i)j )kn can be written as(

∆B(i)j)kn

=(b1 δkj ρ

(i)n + b2 δkn ρ

(i)k + b3 δkj δkn

)ρ(i)j (S13)

whereb1 =

∫ ∞0

∫ ∞0

W3(s1, s2) (�(s1) + �(s2 − s1)) ds1 ds2, (S14)

b2 =∫ ∞

0

∫ ∞0

W3(s1, s2)∫ ∞

0�(r) �(r − s2) dr ds1 ds2, (S15)

andb3 =

∫ ∞0

∫ ∞0

W3(s1, s2) �(s1) �(s2 − s1) ds1 ds2. (S16)

In the presence of P Poisson patterns, we can write the expected weight dynamics as

ẇ =P∑i=1

pi

(φ(ν(i), ν̄)1 + Λ(i)

)ρ(i), (S17)

Supplementary Text 3

whereφ(ν(i), ν̄) = W 2

ν

ρ20ν(i) +W 3

(ν(i))2, (S18)

1 is the identity matrix, and Λ(i) is a diagonal matrix where the jth diagonal element is

Λ(i)jj =(b1wj ν

(i) + b2 ν(i)2 + b3w

2j

)ρ(i)j (S19)

with ν(i)2 =∑

k w2k ρ

(i)k . Recall from the main text that W 2 =

∫ 0−∞ W2(s) ds and W 3 =∫∞

0

∫∞0 W3(s) ds1 ds2; also ν̄ =

∑m pm

(ν(m)

)2with ν(m) = wTρ(m).

If we further assume that the input patterns are orthogonal (and N = P ), then the conditionẇ = 0 implies that φ(ν(i), ν̄(i)) + Λ(i)ii = 0, for all i = 1, . . . , N and therefore

w2i F(ρ(i)i

)−Gρ(i)i wi ν̄ = 0 ∀i = 1, . . . , N (S20)

where

G = −W 2ρ20

> 0 and F(ρ(i)i

)= W 3

(ρ(i)i

)2+ (b1 + b2) ρ

(i)i + b3. (S21)

Each fixed point of Eqn. S17 must satisfy the N conditions from Eqn. S20. Each condition hastwo solutions: either w∗i = 0 or w

∗i = Gρ

(i)i /F

(ρ(i)i

). As a consequence, there are 2N fixed

points, which is consistent with the BCM theory.It remains to be shown that the maximally selective fixed points are stable. The nth fixed

point is given byw∗(n) = (0, . . . , 0, w∗n, 0, . . . , 0)

T

where

w∗n =F(ρ(n)n

)Gpn

(ρ(n)n

)3 takes the nth position.To demonstrate that this fixed point is stable, we have to show that the eigenvalues of theJacobian of Eqn. S17 are negative when evaluated at w∗(n). We find that this Jacobian matrixis given by

Jij

((w∗(n)

)= −δij piG

(ρ(i)i

)2ν̄(n) (S22)

where ν̄(n) = pnw2n(ρ(n)n

)2. Since this matrix is diagonal with all diagonal elements being

negative, we know that all the maximally selective fixed points are stable.Note that around this fixed point, the sliding threshold takes the value

θ = w∗(n)ρ(n)n = θ0(ν̄)(

1 +(τ1ρ

(n)n

)−1+(τ2ρ

(n)n

)−2)−1(S23)

where we recall form the main text that θ0(ν̄) = A−2 τ−ν̄/(A+3 τ+τyρ

p0) is the sliding threshold of

the BCM term only when the correlation terms can be neglected (i.e. ∆A = 0 and ∆B = 0) andthe new timecales are

τ1 = W 3/(b1 + b2) and τ2 =√W 3/b3. (S24)

Supplementary Text 4

One can calculate these values and obtain

W 3 = A+3 τ+τy, (S25)

b1 + b2 =1

τm + τ++

τ+(τm + τ+)(τ+ + τy)

+1

2(τm + τy), (S26)

andb3 =

τ+(τm + 2τ+)(τ+τy + τmτ+ + τmτy)

. (S27)

Selectivity with two correlation-based input patterns

We derive the fixed points of the two-dimensional system of Eqn. 9 for P = 2 patterns withrates ρ(i), pairwise correlations A(i) and third-order correlations B(i) presented to the networkwith probabilities p1 and p2, respectively

ẇ =∑i=1,2

pi

A(i)w∑i=1,2 pi (wTρ(i))2ρ20

+∑k

wTB(i)k w ek

. (S28)To obtain the general fixed points of this equation, we have to simultaneosly solve ẇ1 = 0 andẇ2 = 0. This amounts to solving two cubic equations for which we does not give nice analyticalexpressions. Thus, we looked for fixed points of maximum selectivity (w∗1, 0) and (0, w

∗2) with

w∗1, w∗2 > 0 (ignoring the origin as the trivial fixed point).

To find a fixed point on the w1 axis, we solve ẇ1 = 0 at w1 = w∗1 and w2 = 0

0 = p1

[A

(1)11 w1

p1(wTρ(1)

)2+ p2

(wTρ(2)

)2ρ20

+(B

(1)1

)11w21

]

+ p2

[A

(2)11 w1

p1(wTρ(1)

)2+ p2

(wTρ(2)

)2ρ20

+(B

(2)1

)11w21

](S29)

which gives the following linear equation

(p1A

(1)11 + p2A

(2)11

) p1 (ρ(1)1 )2 + p2 (ρ(2)1 )2ρ20

w∗1 +(p1

(B

(1)1

)11

+ p2(B

(2)1

)11

)= 0. (S30)

This equation has the solution

w∗1 = −p1

(B

(1)1

)11

+ p2(B

(2)1

)11

ρ−20

(p1A

(1)11 + p2A

(2)11

)p1

(ρ(1)1

)2+ p2

(ρ(2)1

)2 (S31)which is given in Eqn. 13.

Supplementary Text 5

Instead of examining the Jacobian of the system at the fixed points to obtain their stability,because of the nonlinearity due to the lower bound on the weights, two conditions must besatisfied for stability:

∂F1(w)∂w1

∣∣∣w=w∗

< 0, (S32)

F2(w∗) < 0. (S33)

The first condition becomes

3(p1A

(1)11 + p2A

(2)11

) p1 (ρ(1)1 )2 + p2 (ρ(2)1 )2ρ20

w∗1 + 2(p1

(B

(1)1

)11

+ p2(B

(2)1

)11

)< 0. (S34)

If we use the expression for w∗1 from Eqn. S31, then the condition reduces to

p1

(B

(1)1

)11

+ p2(B

(2)1

)11> 0 (S35)

which is always true since the correlation terms convolved with the triplet rule in B(i)k are alwayspositive.

The second condition becomes

(p1A

(1)21 + p2A

(2)21

) p1 (ρ(1)1 )2 + p2 (ρ(2)1 )2ρ20

w∗1 +(p1

(B

(1)2

)11

+ p2(B

(2)2

)11

)< 0. (S36)

If we use the expression for w∗1 from Eqn. S31, the condition reduces to

p1A(1)21 + p2A

(2)21

p1A(1)11 + p2A

(2)11

>p1

(B

(1)2

)11

+ p2(B

(2)2

)11

p1

(B

(1)1

)11

+ p2(B

(2)1

)11

. (S37)

Similarly, the fixed point on the w2 axis is stable if the following condition holds

p1A(1)12 + p2A

(2)12

p1A(1)22 + p2A

(2)22

>p1

(B

(1)1

)22

+ p2(B

(2)1

)22

p1

(B

(1)2

)11

+ p2(B

(2)2

)22

. (S38)

Numerical evaluations of these two conditions for a variety of firing rates, pairwise and third-order correlations demonstrated that there conditions always hold. Therefore, in the case of atwo-dimenstional network, the system always results in selectivity. As we show in the main text,this is not always the case for a general N -dimensional system, where selectivity depends on theinput correlation structure.

Supplementary Text 6

Simulations of correlated spike trains.

Higher-order correlations in the spike trains in Figure 5 were simulated by taking the union of anindependent spike train and a common spike train (Gütig et al., 2003; Kuhn et al., 2003; Brette,2009). Applying this method resulted in spike trains with the same rates and same higher-ordercorrelations, with the cross-correlations being delta functions.

This approach was used for the generation of spatial correlations, however, for the spatio-temporal correlations the mixture method described by Brette (2009) was used. In this case,instantaneous correlations were generated by taking the union of an independent spike trainand a common spike train (as in the case of spatial correlations), and then the generated spikeswere shifted by independent and identically distributed random numbers from an appropriatedistribution function. We used an exponential distribution with a time constant τc, resulting inpairwise correlations equal to

Cij(s) = γij/(2τc) e−|s|/τc + ρi ρj (S39)

and a triplet correlations

Ukjn(s1, s2) = Vkjn(s1, s2) + ρk Cjn(s2 − s1) + ρj Ckn(s2) + ρnCkj(s1)− ρk ρj ρn (S40)

where Vkjn(s1, s2) is given by

Vkjn(s1, s2) =βkjn3τ2c

e−(s1+s2)/τc , s1 ≥ 0, s2 ≥ 0e(2s1−s2)/τc , s1 < 0, s2 > s1e(−s1+2s2)/τc , s1 > s2, s2 < 0.

(S41)

Thus, for simplicity we studied symmetric correlation functions, and assumed uniform correla-tions for all inputs pairs and triplets, i.e. the correlation strength was regulated by the coefficientsγij = γ and βkjn = β.

References

Brette, R. (2009). Generation of correlated spike trains. Neural Comp, 21:188–215.

Gütig, R., Aharonov, R., Rotter, S., and Sompolinsky, H. (2003). Learning input correlationsthrough non-linear temporally asymmetric Hebbian plasticity. J Neurosci, 23:3697–3714.

Kuhn, A., Aertsen, A., and Rotter, S. (2003). Higher-order statistics of input ensembles andthe response of simple model neurons. Neural Comp, 15:67–101.