Associative Memory with Dynamic Synapses

LETTER Communicated by Alessandro Treves

Associative Memory with Dynamic Synapses

Lovorka PanticlovorkambfyskunnlJoaqu Acircotilde n J Torrescurren

jtorresonsagerugresHilbert J KappenbertmbfyskunnlStan CAM GielenstanmbfyskunnlDepartment of Biophysics University of Nijmegen 6525 EZ NijmegenThe Netherlands

We have examined a role of dynamic synapses in the stochastic Hopeld-like network behavior Our results demonstrate an appearance of a novelphase characterized by quick transitions from one memory state to an-other The network is able to retrieve memorized patterns correspondingto classical ferromagnetic states but switches between memorized pat-terns with an intermittent type of behavior This phenomenon might re-ect the exibility of real neural systems and their readiness to receiveand respond to novel and changing external stimuli

1 Introduction

Recent experimental studies report that synaptic transmission properties ofcortical cells are strongly inuenced by recent presynaptic activity (AbbottVarela Sen amp Nelson 1997 Tsodyks amp Markram 1997) The postsynapticresponse is activity dependent and can decrease (in case of synaptic de-pression) or increase (in case of synaptic facilitation) under repeated stim-ulation (Tsodyks amp Markram 1997) Markram and Tsodyks (1996) foundthat for some cortical synapses after induction of long-term potentiation(LTP) the temporal synaptic response was not uniformly increased In-stead the amplitude of the initial postsynaptic potential was potentiatedwhereas the steady-state synaptic response was unaffected by LTP (synapticredistribution)

These ndings affect the transmission properties of single neurons aswell as network functioning and behavior Previous studies on the effectsof synaptic depression on network behavior have focused on both feed-forward and recurrent networks For feedforward networks some authors

curren Present address Dept of Electromagnetism and Material Physics University ofGranada E-18071 Granada Spain

Neural Computation 14 2903ndash2923 (2002) cdeg 2002 Massachusetts Institute of Technology

2904 L Pantic J J Torres H J Kappen and SCAM Gielen

have studied a role of activity-dependent synapses in a supervised learningparadigm (Natschlager Maass amp Zador 2001) They showed that even witha single hidden layer of binary neurons such networks can approximate alarge class of nonlinear lters Another study (Liaw amp Berger 1996) demon-strated that dynamically changing synaptic strength allows the extraction ofstatistically signicant features from noisy and variable temporal patternsAn application was presented where a simple neural network with dynamicsynapses was able to perform speech recognition from unprocessed noisywaveforms of words spoken by multiple speakers

For recurrent networks a number of theoretical and numerical stud-ies revealed that a large population of excitatory neurons with depressingsynapses can exhibit complex regimes of activity (Bressloff 1999 Kistleramp van Hemmen 1999 Senn et al 1996 Tsodyks Pawelzik amp Markram1998 Tsodyks Uziel amp Markram 2000) Tsodyks et al (2000) demonstratedthat such networks exhibit short intervals of highly synchronous activity(population bursts) intermittent with long periods of asynchronous activityresembling the behavior observed in neurons throughout the cortex It hasbeen proposed (Senn et al 1996) that synaptic depression may also serveas a mechanism that can provoke rhythmic activity and therefore can havean important role in central pattern generation

The effects of synaptic dynamics on some complex neural functions suchas associative memory are not yet fully established and only a few stud-ies (Bibitchkov Herrmann amp Geisel 2002) have focused on this impor-tant issue The conventional way for model studies of associative memoryfunctions is based on the view that recurrent (attractor) neural networksmay capture some basic characteristics of associative memory (Amit 1989Miyashita amp Chang 1988) In such networks long-term storage of the mem-ory patterns is enabled by synaptic connections that are adjusted (accordingto the Hebb rule) such that the attractors (xed points) of the network dy-namics represent the stored memories (Hopeld 1982) The question onemay ask is how the retrieval properties and the xed points of the Hopeldnetwork are affected by the experimental ndings showing use-dependentweakening of the synaptic connections Bibitchkov et al (2002) reported thatshort-term synaptic depression does not affect the xed points of the net-workbut rather reduces the stability of the patterns and the storage capacityThey also showed that during external stimulation depression mechanismenables easier switching between patterns and better storage of temporalsequences

In this study we aim to extend previous work and explore the role ofnoise as well as the effect of the synaptic recovery process in pattern re-trieval We found that a network with depressing synapses is sensitive tonoise and displays rapid switching among stored memories The switchingbehavior is observed for overlapping as well as nonoverlapping patternsIt exists for the network consisting of simple binary units and also for thenetwork with integrate-and-re neurons The transitions among memoriesstart to occur when the network dynamics described by an iterative (mean-

Associative Memory with Dynamic Synapses 2905

eld) map undergoes an intermittent behavior The memory states lose ab-solute stability and the system instead of residing at one of the xed-pointattractors reveals another type of dynamics Linear stability analysis as wellas numerical simulations reveal that in addition to traditional ferromag-netic (memory) and paramagnetic (noisy) xed-point regimes there existsa limit-cycle phase that enables transitions among the stored memories

2 Models

The classical Hopeld-like neural network consists of N fully connectedbinary neurons The neurons are regarded as stochastic two-state deviceswith the state-variable s taking values 1 (ring) or 0 (no ring) according tothe probabilistic rule given by

Probfsi (t C 1) D 1g D12

[1 C tanh(2bhi (t) ) ] (21)

Parameter b D 1T represents the level of noise caused by synaptic stochas-

ticity or other uctuations in neuron functioning The local eld hi is the netinput to neuron i given by hi D

Pj v0

ijsj iexclhi where v0ij represents the synaptic

connection strength (weights) from neuron j to neuron i The updating ofall neurons can be carried out synchronously (Little dynamics) at each timestep or asynchronously (Glauber dynamics) updating them one at a timeThe connectivity matrixccedil v0

ij is dened according to the standard covariancerule

w0ij D

1Nf (1 iexcl f )

PX

m D1

(jmi iexcl f ) (jm

j iexcl f ) (22)

which represents an optimal learning rule for associative memory networks(Dayan amp Willshaw 1991 Palm amp Sommer 1996) The jm rsquos (m D 1 P)denote P patterns to be memorized (stored) and later to be retrieved bythe network All thresholds hi are set to zero (Peretto 1992) The variable frepresents the mean level of activity of the network for the P patterns It isknown that the memorized patterns are stable xed points of the networkdynamics as long as a acute P

N is small and as long as the noise level is nottoo high (Amit 1989 Peretto 1992) (ferromagnetic phase) When b lt 1 thenoise is stronger than the connectivity and memory is lost (paramagneticphase) When a is too large the interference between patterns destroyspattern stability and causes the spin-glass phase to dominate (Amit 1989)

As a model of the synaptic depression we use the phenomenologicalmodel introduced in Tsodyks and Markram (1997) This model assumesthat synaptic resources are available in a fraction of recovered (x) active (y)and inactive (z) states and that the total amount of resources (neurotrans-mitter) in the recycling process is constant which gives x C y C z D 1 Each


action potential (AP) activates a fraction (U) of the resources in the recov-ered state which (instantaneously) becomes active (during transmission)then becomes inactivated with a time constant (tin) of a few millisecondsand nally recovers with a time constant (trec) that can vary from tens ofmilliseconds to seconds depending on the type of the neuron and the typeof synapse The synaptic current is proportional to the fraction of activeresources I (t) D Ay(t) where A determines the maximal response of thesynapses tin is used to model the a-function like response of the synapseThe model can be further simplied since the inactivation time constant tinis much shorter than the recovery constant trec (Tsodyks et al 1998) Thedetailed synaptic response can be ignored and we can eliminate the vari-able y from the synaptic dynamics Thus we assume that the depressingsynapse can be described by the dynamics of the recovered resources onlySince the states of the binary neurons are updated at discrete time steps weuse a discretized equation

x (t C dt) D x (t) C dtsup3

1 iexcl x(t)trec

iexcl Ux(t)s(t)acute

(23)

instead of the original differential equation (Tsodyks amp Markram 1997Tsodyks et al 1998) with dt denoting the time step of discretization (up-dating)

To incorporate the synaptic dynamics into the network we consider thatduring memory retrieval the strength of synaptic connections (weights)changes in time according to the depressing mechanism The dynamicsynaptic strength vij (t) is assumed to be proportional to the fraction of recov-ered resources xj (t) and the static Hebbian term w0

ij that is wij (t) D w0ijxj (t) 1

The local elds are then given by hi DP

j w0ijxjsj The states of all the neurons

are updated synchronously at time steps ofdt D 1 msec and together denethe consecutive network states The overlap of the instantaneous networkstate with the memorized pattern m that is mm acute 1

Nf (1iexcl f )PN

iD1 (jmi iexcl f )si

provides insight into the network dynamics and measures the similarity ofthe actual state and the stored patterns

3 Simulation Results

31 Network with Binary Units We analyze the behavior of a networkconsisting of N fully connected binary neurons for a small number of pat-terns We rst consider the case of a single pattern with f D 1

2 The patternis stored according to rule 22 and the states of the neurons are governedby equations 21 and 23

1 It is worth mentioning that under this assumption interactions between the neuronsare no longer symmetric


For static synapses it is well known that the network exhibits two typesof behavior The pattern (antipattern) is either permanently retrieved (fer-romagnetic phase) or the retrieval is not possible (paramagnetic phase) Fordepressing synapses the pattern is stable only for a certain period of timeafter which it disappears (see Figure 1 top) All efferent (outgoing) synapsesfrom active units are permanently depressed during this period their aver-age level of recovery xC acute 1

Nf

Pj2Act xj decreases to a low level (see Figure 1

bottom) At the same time all efferent synapses from inactive units recoverthat is xiexcl acute 1

N (1iexcl f )P

j 62Act xj increases Due to noise the network at somemoment of time switches to the attractor formed by the recovered synapses(antipattern) (see Figure 1 top) The antipattern is now temporarily activethen it disappears while the original pattern is being instantaneously reacti-vated and so forth This process repeats itself and Figure 1 (bottom) showsthe repetitive behavior of the xC xiexcl and overlap m1 for two values of trecFor smaller values of trec the duration of the pattern (antipattern) activ-ity is longer than for larger values of trec for which the switching betweenattractor states occurs more frequently

0 100 200 300

0

50

100

0 100 200 300shy 1

shy 05

0

05

1

0 100 200 300

0

50

100

0 100 200 300shy 1

shy 05

0

05

1

b = 20 trec

= 18 msec b = 20 trec

= 30 msec

Figure 1 The periodic regimes of the behavior of an associative memory net-work with dynamic synapses with a single stored pattern ( f D 05) The patternconsists of active (j D 1 60) and inactive (j D 61 120) units (Upperpanels) A raster plot of the neural activity of all neurons (N D 120) in the net-work during 300 msec (Bottom panels) the overlap m1 (thin line) the averagerecovery level of the active units xC (solid thick line) and the average recoverylevel of the inactive units xiexcl(dashed thick line) as a function of time For claritythe dashed thick line is not presented on the right panel


In the case of several orthogonal patterns2 whose active units overlap(see Figure 2 top) the network also displays switching behavior Panelsshowing overlaps with P D 6 patterns indicate that the network rst recallspattern 6 and its antipattern alternately (shown in the sixth panel of thegure) then switches to pattern 3 and its antipattern (third panel) and thenretrieves pattern 5 and its corresponding antipattern (fth panel) Switchingthat occurs on the fast timescale between the pattern and its antipattern issimilar to the behavior shown in Figure 1 (limit cycle) In addition thereexists a switching on the slower timescale among the different patterns Theswitching shown in Figure 2 is more complex than the switching consideredin Figure 1 since one limit cycle switches to another pattern or to anotherlimit cycle This motion might be related to the behavior that can occur forseveral coexisting attractive states

Similar behavior is also present when several patterns with nonover-lapping active units are stored in the network The upper panel of Figure 3shows the activity of the network as a function of time and the middle panelshows the level of recovery xj for neuron j (j D 1 N) All synapses fromtemporarily active units become depressed while synapses from inactiveunits are either recovered or are recovering (see Figure 3 middle) Randomsynchronous activity of some units from an inactive group (which occursdue to noise) will cause an increase of their local elds and further excitationof the other units in the same group (associative mechanisms) Meanwhilethe local elds of the neurons from the currently active group become sup-pressed and the ctivity of that group is inhibited As a result of this processthe network switches to other (mixture) states and so forth

32 Network with Integrate-and-Fire Units We now explore theswitching behavior for the more realistic network with integrate-and-re(IF) neurons We consider N D 200 fully connected neurons whose statesare modeled by the membrane potential Vi (t) (i D 1 N) according tothe dynamics

tmdVi (t) dt D iexclVi (t) C RIsyni (t) C j (t) (31)

The membrane time constant and the membrane resistance are xed totm D 20 msec and R D 100 MV respectively j (t) represents gaussian whitenoise with autocorrelation hj (t)j (t0 ) i D Dd (tiexclt0 ) The synaptic input to cell ifrom other neurons is given by Isyn

i D gAPN

jD1 w0ijyj where Ayj represents the

synaptic current Ij of the neuron j (see section 2) w0ij denotes the connectivity

matrix given by 22 and g is a scaling factor Whenever the membranepotential is equal to the threshold value (Vth D 10 mV) an action potentialis emitted and the potential is reset to zero with the refractoryperiodof tref D

2 1N

PNiD 1

(jmi iexcl f )(jordm

i iexcl f ) D 0 when m 6D ordm


Figure 2 (Top) Raster plot of a network with N D 128 neurons and P D 6orthogonal patterns ( f D 05) The patterns consist of alternating blocks of activeunits (ones) and inactive units (zeros) The rst pattern consists of two blockswith size fN the second pattern consists of 22 blocks with size f 2N the third iscomposed of 23 blocks with size f 3N and so forth The noise level is xed to b D30 and the time constant for recovery is trec D 30 msec Other panels representoverlaps between the current network state and the memorized patterns

1 msec To enable easier observation we store P D 5 patterns whose activebits do not overlap Figure 4 shows the network behavior during the patternretrieval For some xed noise level and depending on the value of the timeconstant for recovery trec the network either permanently recalls one ofthe stored patterns (trec D 0 msecmdashstandard behavior of the network withconstant weights) or switches among patterns (trec D 300 msec and trec D800 msec) or exhibits a paramagnetic type of the behavior (trec D 3000 msec)when retrieval of the patterns is not possible We conclude that the effectsof synaptic depression on the network level are qualitatively the same for


0 100 200 300 400 500

0

50

100

01 02 03 04 05 06 07 08 09 1

50 100 150 200 250 300 350 400 450 500

20

40

60

80

100

A

B

Figure 3 (A) Raster plot of neural activity of a network consisting of N D 100neurons with P D 10 nonoverlapping patterns during the period of 500 msectrec is set to 75 msec Each pattern consists of 10 active and 90 inactive units Therst pattern is dened as j 1

j D 1 8j 2 (1 10) and j 1j D 0 8j 62 (1 10)

the second is given by j 2j D 1 8j 2 (11 20) and j 2

j D 0 8 j 62 (11 20) and so forth (B) Level of recovery xj for neuron j (j D 1 N) White regionsrepresent low levels of recovery (depression) while gray (black) parts indicatethat synapses are recovering (recovered)

the network with IF units as it is for the network with the binary neuronsThe most interesting feature is that synaptic depression destabilizes thestored memories and enables noise-induced transitions from one pattern toanother Ithas been demonstrated (Mueller amp Herz 1999) that the associativeproperties of a network with the IF neurons or with binary neurons aresimilar Our results extend this observation for network with depressingsynapses

4 Mean-Field Calculations

To investigate the switching phenomenon further we analyzed the behaviorof the network with the binary units within the mean-eld framework Forsimplicity we consider the case a 0 When the network is close to pattern


0

100

200

0

100

200

0

100

200

0

100

200

trec

= 0 msec

trec

= 300 msec

trec

= 800 msec

trec

= 3000 msec

time (msec) 100 420

Neurons

Figure 4 From top to bottom panels represents raster plots of neural activityof the network with N D 200 integrate-and-re units and P D 5 patterns (withnonoverlapping active bits) for four different values of trec and for a xed noiselevel of D D 05

m D 1 the local eld is given by

hi DX

j

w0ijxjhsji

D1

Nf (1 iexcl f )

X

j

X

m

(j mi iexcl f ) (jm

j iexcl f )xjhsji

frac141

Nf (1 iexcl f )

X

j

(j 1i iexcl f ) (j 1

j iexcl f )xjhsji (41)

where we have ignored contributions from the other patterns This term

scales asq

PN for random patterns The sum

Pj in the previous expression

can be split in two terms

hi D1

Nf (1 iexcl f )(j 1

i iexcl f )

2

4X

j2Act

(j 1j iexcl f )xjhsji C

X

j 62Act

(j 1j iexcl f )xjhsji

3

5 (42)


where j 2 Act (j 62 Act) denote the active (inactive) neurons in pattern 1We denote by xC the mean level of recovery and by mC acute 1

Nf

Pj2Acthsji the

mean activity of the active units For the inactive units we have xiexcl andmiexcl acute 1

N (1iexcl f )P

j 62Acthsji With these denitions we have m1 D mC iexcl miexcl Thelocal eld is now given by

hi D (j 1i iexcl f ) (xCmC iexcl xiexclmiexcl) (43)

Hence for mC we have

mC (t C 1) D12

f1 C tanh[2b (1 iexcl f ) (xC (t)mC (t) iexcl xiexcl (t)miexcl (t) )]g (44)

and for miexcl

miexcl (t C 1) D12

f1 iexcl tanh[2b f (xC (t)mC (t) iexcl xiexcl (t)miexcl (t) )]g (45)

The dynamics of variables xC and xiexcl is governed by the equations

xC (t C 1) D xC (t) Cmicro

1 iexcl xC (t)trec

iexcl UxC (t)mC (t)para

(46)

xiexcl (t C 1) D xiexcl (t) Cmicro

1 iexcl xiexcl (t)trec

iexcl Uxiexcl (t)miexcl (t)para

(47)

which together with equations 44 and 45 represent the four-dimensionaliterative map that approximately describes the network dynamics

5 Stability Analysis

We analyze the stability of the four-dimensional map for the case f D 12

The xed points are given by the equations

mC D12

f1 C tanh [b (xCmC iexcl xiexclmiexcl) ]g

miexcl D12

f1 iexcl tanh [b (xCmC iexcl xiexclmiexcl) ]g

xC D1

1 C c mC

xiexcl D1

1 C c miexcl (51)

with c D Utrec Let us denote by

Ex0 D Eg (Ex)


the iterative map with Ex D (mC miexcl xC xiexcl) and by

Ex D Excurren

the xed-point solution Linear stability analysis demands linearization ofEg(Ex) near Excurren

x0i D gi ( Excurren) C

X

j

gi ( Excurren )

xj(xj iexcl xcurren

j ) C cent cent cent (52)

The eigenvalues li of the matrix D acute gi ( Excurren )xj

indicate the stability of the

xed-point Excurren If |li | lt 1 8i the solution is stable In Figures 5 and 6 weillustrate the stability of the system (see equations 44 through 47) for twodifferent parameter sets The stability matrix D and the eigenvalues aregiven in the appendix Figure 5 shows that there are two critical (bifurca-tion) values of trec (t1 and t2) where |l|max crosses the value 13 For trec lt t1the system has three xed points where two of them are stable and cor-respond to memory (ferromagnetic) solutions As trec increases the xedpoints approach each other and coalesce near trec D t1 The gure showsthat near this point the system undergoes a qualitative change in behav-ior For t1 lt trec lt t2 stable oscillations occur (limit cycle attractor) Theamplitude of oscillations (black and white circles in the gure) graduallydecreases and nally ceases at t2 At this point the oscillatory behavior is re-placed by a stable (|l|max lt 1) xed point which represents the no-memory(paramagnetic) solution (Hopf bifurcation) In sum we see that the networkdynamics in addition to the xed-point (ferromagnetic and paramagnetic)attractors reveals the limit cycle attractor which enables a periodic kind ofbehavior In Figure 6 we present the stability of the map for a larger valueof b Note that for trec lt t1 there exist ve xed points instead of threexed points The additional xed points are unstable (saddle) and near t1they coalesce with the stable xed points When trec is increased the stableand unstable (saddle) points disappear (a saddle-node type of bifurcation)The network dynamics is illustrated in Figure 7 where the xed points aregiven by the intersections of the diagonal and the map The intersection ofthe map with the diagonal at the value mC D 05 represents an unstable xedpoint The values mC 2 f0 1g are stable xed points in the ferromagneticphase However in the oscillatory regime they are no longer xed pointsThe consecutive states of the network at equal time intervals (black dotsin the gure) show that for trec Agrave t1 (upper left panel) the system revealsa limit-cycle behavior For smaller values of trec the system spends moretime near one of the ferromagnetic xed points When trec is almost equal

3 The crossing that occurs for trec lt 1 is not relevant and represents an approximationartifact since our map is not valid in that range


0

025

05

075

1

0 2 4 6 8 10

m+

t rect 1 t 2

0

05

1

15

2

0 2 4 6 8 10

|li| m

ax

t rect 1 t 2

m+ = 05m+ = 05

Figure 5 (Top) The largest eigenvalue |l|max of the linearized (stability) matrixD for a xed noise level b D 3 as a function of trec (in units of milliseconds)The solid line corresponds to |l|max of the ferromagnetic xed point mC 6D 05(memory solutions) and the dot-dashed line to |l|max of the paramagnetic xedpoint mC D 05 (no-memory solution) t1 and t2 denote the bifurcation pointswhere |l|max crosses the value 1 (Bottom) The solid line represents three xedpoints mC asa functionof trec For trec lt t1 the xed point mC D 05 is anunstablexed point while mC 6D 05 are stable xed points In the t1 lt trec lt t2 regionthere exists only one xed point mC D 05 which is unstable together with a(stable) limit cycle The white (black) circles correspond to maximal (minimal)amplitudes of oscillations emerging in this region For trec gt t2 there is onlyone stable xed point solution (mC D 05)


0

025

05

075

1

0 20 40 60 80

m+

t rect 1 t 2

0

1

2

3

4

0 20 40 60 80

|li| m

ax

t rec

t 1 t 2

m+ = 05m+ = 05m+ = 05

1

2

15 16 17

|li| m

ax

t rec

Figure 6 (Top) The largest eigenvalue |l|max of the linearized (stability) matrixD for a xed level of noise b D 20 as a function of trec The solid line correspondsto |l|max of the ferromagnetic xed points mC 6D 05 (memory solutions) and thedot-dashed line to |l|max of the paramagnetic xed point mC D 05 (no-memorysolution) Note that near the rst bifurcation point t1 there exist two morexed points The additional xed points are unstable and their |l|max values arerepresented by the dashed line At the bifurcation point itself the stable (solidline) and unstable (dashed line) points coalesce and upon further increase oftrec disappear entirely (Bottom) The solid line represents the xed points mC

for different values of trec In the t1 lt trec lt t2 region the circles indicate thepresence of an oscillatory phase with white (black) circles corresponding tomaximal (minimal) amplitudes of oscillations


0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 005 010

002

004

006

008

01

b = 20 trec


= 30 msec

b = 20 trec

= 165 msec

Figure 7 A plot of mC (t C 1) versus mC (t) corresponding to equation 44 forb D 20and for three different values of trec The intersections of the diagonal andthe map (solid line) represent the xed points (see equation 51) The (iterative)dynamics of mC (black dots) indicates that the shape of the limit cycle dependson the value of trec Note that for trec Agrave t1 the dynamics is uniformly slowand after some transient the black dots form the typical limit cycle For lowervalues of trec the concentration of the black dots is highest near the places wherethe ferromagnetic xed points used to emerge This means that the dynamics isslower in these areas of the limit cycle and faster in others The lower right gurerepresents the enlarged left corner of the lower-left gure Note that the systemresides for a long time near the corridors (tangency) between the diagonal andthe map when the value trec is close to the value t1

to t1 the system resides for a long time near one of the ferromagnetic xedpoints In this condition the lower panels in Figure 7 show the existenceof two narrow ldquocorridorsrdquo between the diagonal and the map If the stateof the system is in the neighborhood of these corridors the system residesthere for a long time before it escapes and approaches another corridorSuch a nontrivial dynamics resembles intermittent behavior (Hao 1989)and represents in fact the beginning of a periodic motion (cycle) Figure 8shows that the period of oscillations for the critical t1 is very large This isalso conrmed in numerical simulations of the stochastic system (see Fig-


0

100

200

300

400

500

05 1 15 2 25 3

Per

iod

(iter

atio

ns)

ct rect rec

b =100b = 50b = 10

Figure 8 Period of oscillations as a function of the recovery time constant ratiotrect crec

where t crec denotes the critical value (t1) of the recovery time constant (see

Figures 5 and 6) The three different lines correspond to three different noiselevels of b D 10 50 and 100

ure 1 left) which show long-lasting (frac14 100 ms) activations of pattern andantipattern alternately The duration of these long plateaus of pattern andantipattern activities near the critical t1 depends on the value of the param-eter b (see Figure 8) For the smaller values of b (larger noise level) theresidence periods are shorter and for some critical value of b the saddle-node bifurcation ceases to exist while the intermittency disappears (compareFigure 5)

In Figure 9 we summarize the network behavior in the parameter (b trec)space The ferro- and paramagnetic phases denote the regions with sta-ble behavior that is memory and no-memory (noisy) phases respectivelywhereas the oscillatory region represents the novel periodic type of be-havior The dynamics of the synaptic connections therefore modies thephase diagram and produces macroscopic changes of the networkbehavior


0

5

10

15

20

0 2 4 6 8 10

t rec

b

Paramagnetic phase Oscillatory phase

Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase

Figure 9 (Top) Three regimes of behavior in the parameter (b trec) space trec

is given in units of milliseconds The dashed line corresponds to the transitionbetween the memory (ferromagnetic) and oscillatory phase and represents thepairs of critical b and trec (t1) values The solid line shows the transition betweenthe oscillatory and noisy (paramagnetic) phase with pairs of critical b and t2

values (Bottom) Inset of lower-left part of top gure which shows where theoscillatory phase begins The dot-dashed line is not relevant and represents anapproximation artifact since our model (map) is not valid for trec lt 1


6 Discussion

We show that incorporationof experimentally observed short-term synapticdynamics into a stochastic Hopeld network leads to a novel and complexnetwork behavior Such a network displays bursting activity of memorizedpatterns and fast switching between memories Functionality of synapticdepression affects the stability of memory attractors and enables exibletransitions among them The attractors display a kind of metastability wherea small amount of noise can trigger a jump from one state to another Thenetwork dynamics exhibits an oscillatory activity and retrieves patterns ortheir mixtures alternately Although the functional roles and relevance ofthese results for associative memory are not yet clear they might be linkedto the spatiotemporal encoding scheme (Hoshino Kashimori amp Kambara1998) of the olfactory cortex which is believed to represent a good modelfor analysis of associative memory processes (Haberly amp Bower 1989) InHoshino et al (1998) the model based on experimental ndings showedthat the information about odors could be encoded into a spatiotemporalpattern of neural activity in the olfactory bulb which consists of a temporalsequence of spatial activity patterns The recognition of olfactory informa-tion as also suggested by other authors might be related to oscillatoryactiv-ity patterns The rapid switching among memory states and the sensitivityto noise resemble the exibility and metastability of real systems (Kelso1995) in receiving and processing new data in continuously changing exter-nal conditions They may also serve as a possible mechanism underlyingattention or short-term and working memory (Nakahara amp Doya 1998)Nakahara and Doya (1998) emphasized that neural dynamics in workingmemory for goal-directed behaviors should have the properties of long-term maintenance and quick transition It was shown that both propertiescan be achieved when the system parameters are near a saddle-node bi-furcation (point) which itself may be a functional necessity for survival innonstationary environments

A similar switching phenomenon has been reported by Horn and Usher(1989) where threshold dynamics instead of synaptic dynamics has beenconsidered Using the threshold function associated with fatigue of a neu-ron a self-driven temporal sequence of patterns was obtained Both adap-tive mechanisms the dynamic threshold and synaptic depression producesimilar effects on the network behavior and enable transitions among storedmemories One may wonder whether these two mechanisms are formallyequivalent In order to explore this question in more detail consider themean-eld equations mi D tanh(

Pj wijxjmj C hi) The informal equivalence

of dynamic synapses and thresholds is that for dynamic thresholds in-creasing mi will decrease hi which decreases future mi Similarly for dy-namic synapses increased mj will decrease xj reducing the inuence of theproduct xjmj on the network Therefore the net effect is similar and in-


deed dynamic thresholds also show oscillations However we think thatthere is no formal equivalence The reason is that changes in xj affect theslope of mj at the right-hand side of the mean-eld equations thereby pos-sibly changing the number of xed points in the system and dynamicallychanging the system from ferromagnetic to paramagnetic or vice versaChanges in hi cannot bring about such qualitative changes These mecha-nisms as well as some other biological mechanisms (Foss Moss amp Milton1997) which also enable noise-induced transitions among attractors maybe related to more dynamic processes of associative memory the processof associative thinking (Horn amp Usher 1989) or dynamic memory storage(Foss et al 1997)

Our numerical studies of a network consisting of IF neurons with dy-namic synapses show the same switching capability among the memorystates as the network with binary neurons In order to determine the rele-vance of this behavior for neurobiology further analyses of the switchingphenomenon and the phenomenon of intermittency will require more real-istic network models as well as more realistic learning rules

Regarding the learning rule it has been shown that the covariance learn-ing rule provides optimal memory performance of an associative-memorynetwork with binary neurons (Dayan amp Willshaw 1991 Palm amp Sommer1996) However like all other Hebbian forms of plasticity this rule leadsto excessive growth of synaptic strength Recent experimental results sug-gest that there are several regulating mechanisms such as synaptic scalingspike-timing dependent synaptic plasticity (STDP) or synaptic redistribu-tion (Abbott amp Nelson 2000 Bi amp Poo 1998 Markram amp Tsodyks 1996Turrigiano Leslie Desai Rutherford amp Nelson 1998) that can prevent un-controlled synaptic runaway Chechik Meilijson and Ruppin (2001) havealready demonstrated that a kind of synaptic scaling that is a neuronalregulating mechanism maintaining the postsynaptic activity near a xedbaseline improves network memory capacity The effects of these regulat-ing mechanisms on the network behavior reported here need to be exploredfurther

In this study we have focused on a small number of patterns and onthe basic mechanism that enables rapid switching among memories Thesimulations have shown that noise-induced rapid transitions occur amongthe (non)overlapping patterns or the mixture states The transitions becomemore complex when the number of patterns is increased and the behaviorresembles the motion in a complex multistable system with a large numberof coexisting states An important point is whether this behavior would berobust to interference among patterns and how it would affect storage ca-pacity Bibitchkov et al (2002) have shown that the capacity of an innitenetwork decreases with increasing depression using an equilibrium formu-lation of the system Whether this conclusion is also valid for more realisticneuronal and synaptic dynamics needs to be explored further


Appendix Stability Matrix

For the system 44 through 47 and using the stationary conditions 51 thematrix D becomes

D D

0

BBB

2mC (1iexclmC )bxC iexcl2mC (1iexclmC )bxiexcl 2m2C (1iexclmC )b iexcl2mC (1iexclmC )2b

iexcl2mC (1iexclmC )bxC 2mC (1iexclmC )bxiexcl iexcl2m2C (1iexclmC )b 2mC (1iexclmC )2b

iexclUxC 0 1iexcl 1trec

iexclUmC 0

0 iexclUxiexcl 0 1iexcl 1trec

iexclU (1iexclmC )

1

CCCA (A1)

To obtain the eigenvalues of the linearized system we have to solve thecharacteristic equation

|D iexcl lI| D 0 (A2)

where | cent | refers to the determinant and I is the identity matrix This gives oneeigenvalue l1 equal to zero The other three eigenvalues are the solutionsof the cubic equation

shyshyshyshyshyshyshy

2mC (1 iexcl mC )b (xC C xiexcl ) iexcl l iexcl2m2C (1 iexcl mC )b 2mC (1 iexcl mC )2b

UxC 1 iexcl 1trec

iexcl UmC iexcl l 0

iexclUxiexcl 0 1iexcl 1trec

iexcl U (1 iexcl mC ) iexcl l

shyshyshyshyshyshyshyD 0 (A3)

In general equation A3must be solved numerically for an arbitrary solutionmC For the particular solution mC D 12 equation A3 reduces to

sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)

and the other two eigenvalues are the solutions of the quadratic equationsup3

1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments

JJT acknowledges support from MCyT and FEDER (Ram Acircon y Cajal pro-gram) and partial support from Neuroinformatics Thematic Network Uni-versidad de Granada (Plan propio) MCyT (under project BFM2001-2841)and the Dutch Technology Foundation (STW)


References

Abbott L amp Nelson S (2000) Synaptic plasticity Taming the beast NatNeurosci 3 1178ndash1183

Abbott L Varela J Sen K amp Nelson S (1997) Synaptic depression andcortical gain control Science 275 220ndash224

Amit D (1989) Modeling brain function Cambridge Cambridge UniversityPress

Bi G Q amp Poo M M (1998) Activity-induced synaptic modications inhippocampal culture dependence on spike timing synaptic strength andcell type Journal of Neuroscience 18 10464ndash10472

Bibitchkov D Herrmann J M amp Geisel T (2002) Pattern storage and pro-cessing in attractor networks with short-time synaptic dynamics NetworkComput Neural Syst 13(1) 115ndash129

Bressloff P C (1999)Dynamic synapses a new concept of neural representationand computation Phys Rev E 60(2) 2160ndash2170

Chechik G Meilijson I amp Ruppin E (2001) Effective neuronal learning withineffective Hebbian learning rules Neural Comput 13(4) 817ndash840

Dayan P amp Willshaw D (1991) Optimizing synaptic learning rules in linearassociative memories Biol Cyber 65(4) 253ndash265

Foss J Moss F amp Milton J (1997) Noise multistability and delayed recurrentloops Phys Rev E 55(4) 4536ndash4543

Haberly L B amp Bower J M (1989) Olfactory cortex model circuit for studyof associative memory Trends Neurosci 12 258ndash264

Hao B L (1989) Elementary symbolic dynamics Singapore World ScienticPublishing

Hopeld J J (1982) Neural networks and physical systems with emer-gent collective computational abilities Proc Natl Acad Sci USA 79 2554ndash2558

Horn D amp Usher M (1989) Neural networks with dynamical thresholdsPhysical Review A 40(2) 1036ndash1044

Hoshino K Kashimori Y amp Kambara T (1998) An olfactory recognitionmodel based on spatio-temporal encoding of odor quality in the olfactorybulb Biol Cybern 79 109ndash120

Kelso J S (1995) Dynamic patterns the self-organization of brain and behaviorCambridge MA MIT Press

Kistler W M amp van Hemmen J L (1999) Short-term synaptic plasticity andnetwork behavior Neural Comput 11(7) 1579ndash1594

Liaw J S amp Berger T W (1996) Dynamic synapses a new concept of neuralrepresentation and computation Hippocampus 6 591ndash600

Markram H amp Tsodyks M (1996) Redistribution of synaptic efcacy betweenneocortical pyramidal neurons Nature 382 807ndash810

Miyashita Y amp Chang H S (1988) Neuronal correlate of pictorial short-termmemory in the primate temporal cortex Nature 331(6151) 68ndash70

Mueller R amp Herz V M (1999) Content-addressable memory with spikingneurons Phys Rev E 59(3) 3330ndash3338


Nakahara H amp Doya K (1998) Near saddle-node bifurcation behavior asdynamics in working memory for foal-directed behavior Neural Comput10(1) 113ndash132

Natschlaumlger T Maass W amp Zador A (2001) Efcient temporal processingwith biologically realistic dynamic synapses Network Computation in NeuralSystems 12(1) 75ndash87

Palm G amp Sommer F (1996) Models of neural networks III Berlin Springer-Verlag

Peretto P (1992) An introduction to the modeling of neural networks CambridgeCambridge University Press

Senn W Segev I amp Tsodyks M (1998) Reading neuronal synchrony withdepressing synapse Neural Comput 10(4) 815ndash819

Senn W Wyler K Streit J Larkum M Luscher H R Mey H Muller LStainhauser D Vogt K amp Wannier T (1996) Dynamics of a random neuralnetwork with synaptic depression Neural Networks 9(4) 575ndash588

Tsodyks M amp Markram H (1997) The neural code between neocortical pyra-midal neurons depends on neurotransmitter release probability Proc NatlAcad Sci USA 94 719ndash723

Tsodyks M Pawelzik K amp Markram H (1998) Neural networks with dy-namic synapses Neural Comput 10(4) 821ndash835

Tsodyks M Uziel A amp Markram H (2000) Synchrony generation in recurrentnetworks with frequency-dependent synapses J Neurosci 20(1) 50

Turrigiano G G Leslie K R Desai N S Rutherford L C amp Nelson S B(1998) Activity-dependent scaling of quantal amplitude in neocortical neu-rons Nature 391 892ndash896

Received July 12 2001 accepted May 14 2002


have studied a role of activity-dependent synapses in a supervised learningparadigm (Natschlager Maass amp Zador 2001) They showed that even witha single hidden layer of binary neurons such networks can approximate alarge class of nonlinear lters Another study (Liaw amp Berger 1996) demon-strated that dynamically changing synaptic strength allows the extraction ofstatistically signicant features from noisy and variable temporal patternsAn application was presented where a simple neural network with dynamicsynapses was able to perform speech recognition from unprocessed noisywaveforms of words spoken by multiple speakers

For recurrent networks a number of theoretical and numerical stud-ies revealed that a large population of excitatory neurons with depressingsynapses can exhibit complex regimes of activity (Bressloff 1999 Kistleramp van Hemmen 1999 Senn et al 1996 Tsodyks Pawelzik amp Markram1998 Tsodyks Uziel amp Markram 2000) Tsodyks et al (2000) demonstratedthat such networks exhibit short intervals of highly synchronous activity(population bursts) intermittent with long periods of asynchronous activityresembling the behavior observed in neurons throughout the cortex It hasbeen proposed (Senn et al 1996) that synaptic depression may also serveas a mechanism that can provoke rhythmic activity and therefore can havean important role in central pattern generation

The effects of synaptic dynamics on some complex neural functions suchas associative memory are not yet fully established and only a few stud-ies (Bibitchkov Herrmann amp Geisel 2002) have focused on this impor-tant issue The conventional way for model studies of associative memoryfunctions is based on the view that recurrent (attractor) neural networksmay capture some basic characteristics of associative memory (Amit 1989Miyashita amp Chang 1988) In such networks long-term storage of the mem-ory patterns is enabled by synaptic connections that are adjusted (accordingto the Hebb rule) such that the attractors (xed points) of the network dy-namics represent the stored memories (Hopeld 1982) The question onemay ask is how the retrieval properties and the xed points of the Hopeldnetwork are affected by the experimental ndings showing use-dependentweakening of the synaptic connections Bibitchkov et al (2002) reported thatshort-term synaptic depression does not affect the xed points of the net-workbut rather reduces the stability of the patterns and the storage capacityThey also showed that during external stimulation depression mechanismenables easier switching between patterns and better storage of temporalsequences

In this study we aim to extend previous work and explore the role ofnoise as well as the effect of the synaptic recovery process in pattern re-trieval We found that a network with depressing synapses is sensitive tonoise and displays rapid switching among stored memories The switchingbehavior is observed for overlapping as well as nonoverlapping patternsIt exists for the network consisting of simple binary units and also for thenetwork with integrate-and-re neurons The transitions among memoriesstart to occur when the network dynamics described by an iterative (mean-



2 Models



[1 C tanh(2bhi (t) ) ] (21)



Pj v0




w0ij D

1Nf (1 iexcl f )

PX

m D1

(jmi iexcl f ) (jm

j iexcl f ) (22)







1 iexcl x(t)trec


(23)







Nf (1iexcl f )PN









Nf



N (1iexcl f )P


0 100 200 300

0

50

100

0 100 200 300shy 1

shy 05

0

05

1

0 100 200 300

0

50

100

0 100 200 300shy 1

shy 05

0

05

1

b = 20 trec


= 30 msec








i D gAPN




2 1N

PNiD 1







0 100 200 300 400 500

0

50

100

01 02 03 04 05 06 07 08 09 1

50 100 150 200 250 300 350 400 450 500

20

40

60

80

100

A

B


j D 1 8j 2 (1 10) and j 1j D 0 8j 62 (1 10)







0

100

200

0

100

200

0

100

200

0

100

200

trec

= 0 msec

trec

= 300 msec

trec

= 800 msec

trec

= 3000 msec

time (msec) 100 420

Neurons



hi DX

j

w0ijxjhsji

D1

Nf (1 iexcl f )

X

j

X

m

(j mi iexcl f ) (jm

j iexcl f )xjhsji

frac141

Nf (1 iexcl f )

X

j




scales asq




hi D1

Nf (1 iexcl f )(j 1

i iexcl f )

2

4X

j2Act


X

j 62Act


3

5 (42)



Nf

Pj2Acthsji the


N (1iexcl f )P




mC (t C 1) D12


and for miexcl

miexcl (t C 1) D12




1 iexcl xC (t)trec


(46)




(47)





mC D12


miexcl D12


xC D1

1 C c mC

xiexcl D1

1 C c miexcl (51)


Ex0 D Eg (Ex)



Ex D Excurren



X

j

gi ( Excurren )

xj(xj iexcl xcurren







0

025

05

075

1

0 2 4 6 8 10

m+

t rect 1 t 2

0

05

1

15

2

0 2 4 6 8 10

|li| m

ax

t rect 1 t 2

m+ = 05m+ = 05



0

025

05

075

1

0 20 40 60 80

m+

t rect 1 t 2

0

1

2

3

4

0 20 40 60 80

|li| m

ax

t rec

t 1 t 2

m+ = 05m+ = 05m+ = 05

1

2

15 16 17

|li| m

ax

t rec




0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 005 010

002

004

006

008

01

b = 20 trec


= 30 msec

b = 20 trec

= 165 msec




0

100

200

300

400

500

05 1 15 2 25 3

Per

iod

(iter

atio

ns)

ct rect rec

b =100b = 50b = 10







0

5

10

15

20

0 2 4 6 8 10

t rec

b


Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase





6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References



































2 Models



[1 C tanh(2bhi (t) ) ] (21)



Pj v0




w0ij D

1Nf (1 iexcl f )

PX

m D1

(jmi iexcl f ) (jm

j iexcl f ) (22)







1 iexcl x(t)trec


(23)







Nf (1iexcl f )PN









Nf



N (1iexcl f )P


0 100 200 300

0

50

100

0 100 200 300shy 1

shy 05

0

05

1

0 100 200 300

0

50

100

0 100 200 300shy 1

shy 05

0

05

1

b = 20 trec


= 30 msec








i D gAPN




2 1N

PNiD 1







0 100 200 300 400 500

0

50

100

01 02 03 04 05 06 07 08 09 1

50 100 150 200 250 300 350 400 450 500

20

40

60

80

100

A

B


j D 1 8j 2 (1 10) and j 1j D 0 8j 62 (1 10)







0

100

200

0

100

200

0

100

200

0

100

200

trec

= 0 msec

trec

= 300 msec

trec

= 800 msec

trec

= 3000 msec

time (msec) 100 420

Neurons



hi DX

j

w0ijxjhsji

D1

Nf (1 iexcl f )

X

j

X

m

(j mi iexcl f ) (jm

j iexcl f )xjhsji

frac141

Nf (1 iexcl f )

X

j




scales asq




hi D1

Nf (1 iexcl f )(j 1

i iexcl f )

2

4X

j2Act


X

j 62Act


3

5 (42)



Nf

Pj2Acthsji the


N (1iexcl f )P




mC (t C 1) D12


and for miexcl

miexcl (t C 1) D12




1 iexcl xC (t)trec


(46)




(47)





mC D12


miexcl D12


xC D1

1 C c mC

xiexcl D1

1 C c miexcl (51)


Ex0 D Eg (Ex)



Ex D Excurren



X

j

gi ( Excurren )

xj(xj iexcl xcurren







0

025

05

075

1

0 2 4 6 8 10

m+

t rect 1 t 2

0

05

1

15

2

0 2 4 6 8 10

|li| m

ax

t rect 1 t 2

m+ = 05m+ = 05



0

025

05

075

1

0 20 40 60 80

m+

t rect 1 t 2

0

1

2

3

4

0 20 40 60 80

|li| m

ax

t rec

t 1 t 2

m+ = 05m+ = 05m+ = 05

1

2

15 16 17

|li| m

ax

t rec




0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 005 010

002

004

006

008

01

b = 20 trec


= 30 msec

b = 20 trec

= 165 msec




0

100

200

300

400

500

05 1 15 2 25 3

Per

iod

(iter

atio

ns)

ct rect rec

b =100b = 50b = 10







0

5

10

15

20

0 2 4 6 8 10

t rec

b


Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase





6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References




































1 iexcl x(t)trec


(23)







Nf (1iexcl f )PN









Nf



N (1iexcl f )P


0 100 200 300

0

50

100

0 100 200 300shy 1

shy 05

0

05

1

0 100 200 300

0

50

100

0 100 200 300shy 1

shy 05

0

05

1

b = 20 trec


= 30 msec








i D gAPN




2 1N

PNiD 1







0 100 200 300 400 500

0

50

100

01 02 03 04 05 06 07 08 09 1

50 100 150 200 250 300 350 400 450 500

20

40

60

80

100

A

B


j D 1 8j 2 (1 10) and j 1j D 0 8j 62 (1 10)







0

100

200

0

100

200

0

100

200

0

100

200

trec

= 0 msec

trec

= 300 msec

trec

= 800 msec

trec

= 3000 msec

time (msec) 100 420

Neurons



hi DX

j

w0ijxjhsji

D1

Nf (1 iexcl f )

X

j

X

m

(j mi iexcl f ) (jm

j iexcl f )xjhsji

frac141

Nf (1 iexcl f )

X

j




scales asq




hi D1

Nf (1 iexcl f )(j 1

i iexcl f )

2

4X

j2Act


X

j 62Act


3

5 (42)



Nf

Pj2Acthsji the


N (1iexcl f )P




mC (t C 1) D12


and for miexcl

miexcl (t C 1) D12




1 iexcl xC (t)trec


(46)




(47)





mC D12


miexcl D12


xC D1

1 C c mC

xiexcl D1

1 C c miexcl (51)


Ex0 D Eg (Ex)



Ex D Excurren



X

j

gi ( Excurren )

xj(xj iexcl xcurren







0

025

05

075

1

0 2 4 6 8 10

m+

t rect 1 t 2

0

05

1

15

2

0 2 4 6 8 10

|li| m

ax

t rect 1 t 2

m+ = 05m+ = 05



0

025

05

075

1

0 20 40 60 80

m+

t rect 1 t 2

0

1

2

3

4

0 20 40 60 80

|li| m

ax

t rec

t 1 t 2

m+ = 05m+ = 05m+ = 05

1

2

15 16 17

|li| m

ax

t rec




0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 005 010

002

004

006

008

01

b = 20 trec


= 30 msec

b = 20 trec

= 165 msec




0

100

200

300

400

500

05 1 15 2 25 3

Per

iod

(iter

atio

ns)

ct rect rec

b =100b = 50b = 10







0

5

10

15

20

0 2 4 6 8 10

t rec

b


Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase





6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References



































Nf



N (1iexcl f )P


0 100 200 300

0

50

100

0 100 200 300shy 1

shy 05

0

05

1

0 100 200 300

0

50

100

0 100 200 300shy 1

shy 05

0

05

1

b = 20 trec


= 30 msec








i D gAPN




2 1N

PNiD 1







0 100 200 300 400 500

0

50

100

01 02 03 04 05 06 07 08 09 1

50 100 150 200 250 300 350 400 450 500

20

40

60

80

100

A

B


j D 1 8j 2 (1 10) and j 1j D 0 8j 62 (1 10)







0

100

200

0

100

200

0

100

200

0

100

200

trec

= 0 msec

trec

= 300 msec

trec

= 800 msec

trec

= 3000 msec

time (msec) 100 420

Neurons



hi DX

j

w0ijxjhsji

D1

Nf (1 iexcl f )

X

j

X

m

(j mi iexcl f ) (jm

j iexcl f )xjhsji

frac141

Nf (1 iexcl f )

X

j




scales asq




hi D1

Nf (1 iexcl f )(j 1

i iexcl f )

2

4X

j2Act


X

j 62Act


3

5 (42)



Nf

Pj2Acthsji the


N (1iexcl f )P




mC (t C 1) D12


and for miexcl

miexcl (t C 1) D12




1 iexcl xC (t)trec


(46)




(47)





mC D12


miexcl D12


xC D1

1 C c mC

xiexcl D1

1 C c miexcl (51)


Ex0 D Eg (Ex)



Ex D Excurren



X

j

gi ( Excurren )

xj(xj iexcl xcurren







0

025

05

075

1

0 2 4 6 8 10

m+

t rect 1 t 2

0

05

1

15

2

0 2 4 6 8 10

|li| m

ax

t rect 1 t 2

m+ = 05m+ = 05



0

025

05

075

1

0 20 40 60 80

m+

t rect 1 t 2

0

1

2

3

4

0 20 40 60 80

|li| m

ax

t rec

t 1 t 2

m+ = 05m+ = 05m+ = 05

1

2

15 16 17

|li| m

ax

t rec




0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 005 010

002

004

006

008

01

b = 20 trec


= 30 msec

b = 20 trec

= 165 msec




0

100

200

300

400

500

05 1 15 2 25 3

Per

iod

(iter

atio

ns)

ct rect rec

b =100b = 50b = 10







0

5

10

15

20

0 2 4 6 8 10

t rec

b


Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase





6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References







































i D gAPN




2 1N

PNiD 1







0 100 200 300 400 500

0

50

100

01 02 03 04 05 06 07 08 09 1

50 100 150 200 250 300 350 400 450 500

20

40

60

80

100

A

B


j D 1 8j 2 (1 10) and j 1j D 0 8j 62 (1 10)







0

100

200

0

100

200

0

100

200

0

100

200

trec

= 0 msec

trec

= 300 msec

trec

= 800 msec

trec

= 3000 msec

time (msec) 100 420

Neurons



hi DX

j

w0ijxjhsji

D1

Nf (1 iexcl f )

X

j

X

m

(j mi iexcl f ) (jm

j iexcl f )xjhsji

frac141

Nf (1 iexcl f )

X

j




scales asq




hi D1

Nf (1 iexcl f )(j 1

i iexcl f )

2

4X

j2Act


X

j 62Act


3

5 (42)



Nf

Pj2Acthsji the


N (1iexcl f )P




mC (t C 1) D12


and for miexcl

miexcl (t C 1) D12




1 iexcl xC (t)trec


(46)




(47)





mC D12


miexcl D12


xC D1

1 C c mC

xiexcl D1

1 C c miexcl (51)


Ex0 D Eg (Ex)



Ex D Excurren



X

j

gi ( Excurren )

xj(xj iexcl xcurren







0

025

05

075

1

0 2 4 6 8 10

m+

t rect 1 t 2

0

05

1

15

2

0 2 4 6 8 10

|li| m

ax

t rect 1 t 2

m+ = 05m+ = 05



0

025

05

075

1

0 20 40 60 80

m+

t rect 1 t 2

0

1

2

3

4

0 20 40 60 80

|li| m

ax

t rec

t 1 t 2

m+ = 05m+ = 05m+ = 05

1

2

15 16 17

|li| m

ax

t rec




0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 005 010

002

004

006

008

01

b = 20 trec


= 30 msec

b = 20 trec

= 165 msec




0

100

200

300

400

500

05 1 15 2 25 3

Per

iod

(iter

atio

ns)

ct rect rec

b =100b = 50b = 10







0

5

10

15

20

0 2 4 6 8 10

t rec

b


Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase





6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References





































0 100 200 300 400 500

0

50

100

01 02 03 04 05 06 07 08 09 1

50 100 150 200 250 300 350 400 450 500

20

40

60

80

100

A

B


j D 1 8j 2 (1 10) and j 1j D 0 8j 62 (1 10)







0

100

200

0

100

200

0

100

200

0

100

200

trec

= 0 msec

trec

= 300 msec

trec

= 800 msec

trec

= 3000 msec

time (msec) 100 420

Neurons



hi DX

j

w0ijxjhsji

D1

Nf (1 iexcl f )

X

j

X

m

(j mi iexcl f ) (jm

j iexcl f )xjhsji

frac141

Nf (1 iexcl f )

X

j




scales asq




hi D1

Nf (1 iexcl f )(j 1

i iexcl f )

2

4X

j2Act


X

j 62Act


3

5 (42)



Nf

Pj2Acthsji the


N (1iexcl f )P




mC (t C 1) D12


and for miexcl

miexcl (t C 1) D12




1 iexcl xC (t)trec


(46)




(47)





mC D12


miexcl D12


xC D1

1 C c mC

xiexcl D1

1 C c miexcl (51)


Ex0 D Eg (Ex)



Ex D Excurren



X

j

gi ( Excurren )

xj(xj iexcl xcurren







0

025

05

075

1

0 2 4 6 8 10

m+

t rect 1 t 2

0

05

1

15

2

0 2 4 6 8 10

|li| m

ax

t rect 1 t 2

m+ = 05m+ = 05



0

025

05

075

1

0 20 40 60 80

m+

t rect 1 t 2

0

1

2

3

4

0 20 40 60 80

|li| m

ax

t rec

t 1 t 2

m+ = 05m+ = 05m+ = 05

1

2

15 16 17

|li| m

ax

t rec




0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 005 010

002

004

006

008

01

b = 20 trec


= 30 msec

b = 20 trec

= 165 msec




0

100

200

300

400

500

05 1 15 2 25 3

Per

iod

(iter

atio

ns)

ct rect rec

b =100b = 50b = 10







0

5

10

15

20

0 2 4 6 8 10

t rec

b


Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase





6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References


































0 100 200 300 400 500

0

50

100

01 02 03 04 05 06 07 08 09 1

50 100 150 200 250 300 350 400 450 500

20

40

60

80

100

A

B


j D 1 8j 2 (1 10) and j 1j D 0 8j 62 (1 10)







0

100

200

0

100

200

0

100

200

0

100

200

trec

= 0 msec

trec

= 300 msec

trec

= 800 msec

trec

= 3000 msec

time (msec) 100 420

Neurons



hi DX

j

w0ijxjhsji

D1

Nf (1 iexcl f )

X

j

X

m

(j mi iexcl f ) (jm

j iexcl f )xjhsji

frac141

Nf (1 iexcl f )

X

j




scales asq




hi D1

Nf (1 iexcl f )(j 1

i iexcl f )

2

4X

j2Act


X

j 62Act


3

5 (42)



Nf

Pj2Acthsji the


N (1iexcl f )P




mC (t C 1) D12


and for miexcl

miexcl (t C 1) D12




1 iexcl xC (t)trec


(46)




(47)





mC D12


miexcl D12


xC D1

1 C c mC

xiexcl D1

1 C c miexcl (51)


Ex0 D Eg (Ex)



Ex D Excurren



X

j

gi ( Excurren )

xj(xj iexcl xcurren







0

025

05

075

1

0 2 4 6 8 10

m+

t rect 1 t 2

0

05

1

15

2

0 2 4 6 8 10

|li| m

ax

t rect 1 t 2

m+ = 05m+ = 05



0

025

05

075

1

0 20 40 60 80

m+

t rect 1 t 2

0

1

2

3

4

0 20 40 60 80

|li| m

ax

t rec

t 1 t 2

m+ = 05m+ = 05m+ = 05

1

2

15 16 17

|li| m

ax

t rec




0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 005 010

002

004

006

008

01

b = 20 trec


= 30 msec

b = 20 trec

= 165 msec




0

100

200

300

400

500

05 1 15 2 25 3

Per

iod

(iter

atio

ns)

ct rect rec

b =100b = 50b = 10







0

5

10

15

20

0 2 4 6 8 10

t rec

b


Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase





6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References


































0

100

200

0

100

200

0

100

200

0

100

200

trec

= 0 msec

trec

= 300 msec

trec

= 800 msec

trec

= 3000 msec

time (msec) 100 420

Neurons



hi DX

j

w0ijxjhsji

D1

Nf (1 iexcl f )

X

j

X

m

(j mi iexcl f ) (jm

j iexcl f )xjhsji

frac141

Nf (1 iexcl f )

X

j




scales asq




hi D1

Nf (1 iexcl f )(j 1

i iexcl f )

2

4X

j2Act


X

j 62Act


3

5 (42)



Nf

Pj2Acthsji the


N (1iexcl f )P




mC (t C 1) D12


and for miexcl

miexcl (t C 1) D12




1 iexcl xC (t)trec


(46)




(47)





mC D12


miexcl D12


xC D1

1 C c mC

xiexcl D1

1 C c miexcl (51)


Ex0 D Eg (Ex)



Ex D Excurren



X

j

gi ( Excurren )

xj(xj iexcl xcurren







0

025

05

075

1

0 2 4 6 8 10

m+

t rect 1 t 2

0

05

1

15

2

0 2 4 6 8 10

|li| m

ax

t rect 1 t 2

m+ = 05m+ = 05



0

025

05

075

1

0 20 40 60 80

m+

t rect 1 t 2

0

1

2

3

4

0 20 40 60 80

|li| m

ax

t rec

t 1 t 2

m+ = 05m+ = 05m+ = 05

1

2

15 16 17

|li| m

ax

t rec




0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 005 010

002

004

006

008

01

b = 20 trec


= 30 msec

b = 20 trec

= 165 msec




0

100

200

300

400

500

05 1 15 2 25 3

Per

iod

(iter

atio

ns)

ct rect rec

b =100b = 50b = 10







0

5

10

15

20

0 2 4 6 8 10

t rec

b


Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase





6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References



































Nf

Pj2Acthsji the


N (1iexcl f )P




mC (t C 1) D12


and for miexcl

miexcl (t C 1) D12




1 iexcl xC (t)trec


(46)




(47)





mC D12


miexcl D12


xC D1

1 C c mC

xiexcl D1

1 C c miexcl (51)


Ex0 D Eg (Ex)



Ex D Excurren



X

j

gi ( Excurren )

xj(xj iexcl xcurren







0

025

05

075

1

0 2 4 6 8 10

m+

t rect 1 t 2

0

05

1

15

2

0 2 4 6 8 10

|li| m

ax

t rect 1 t 2

m+ = 05m+ = 05



0

025

05

075

1

0 20 40 60 80

m+

t rect 1 t 2

0

1

2

3

4

0 20 40 60 80

|li| m

ax

t rec

t 1 t 2

m+ = 05m+ = 05m+ = 05

1

2

15 16 17

|li| m

ax

t rec




0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 005 010

002

004

006

008

01

b = 20 trec


= 30 msec

b = 20 trec

= 165 msec




0

100

200

300

400

500

05 1 15 2 25 3

Per

iod

(iter

atio

ns)

ct rect rec

b =100b = 50b = 10







0

5

10

15

20

0 2 4 6 8 10

t rec

b


Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase





6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References



































Ex D Excurren



X

j

gi ( Excurren )

xj(xj iexcl xcurren







0

025

05

075

1

0 2 4 6 8 10

m+

t rect 1 t 2

0

05

1

15

2

0 2 4 6 8 10

|li| m

ax

t rect 1 t 2

m+ = 05m+ = 05



0

025

05

075

1

0 20 40 60 80

m+

t rect 1 t 2

0

1

2

3

4

0 20 40 60 80

|li| m

ax

t rec

t 1 t 2

m+ = 05m+ = 05m+ = 05

1

2

15 16 17

|li| m

ax

t rec




0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 005 010

002

004

006

008

01

b = 20 trec


= 30 msec

b = 20 trec

= 165 msec




0

100

200

300

400

500

05 1 15 2 25 3

Per

iod

(iter

atio

ns)

ct rect rec

b =100b = 50b = 10







0

5

10

15

20

0 2 4 6 8 10

t rec

b


Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase





6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References


































0

025

05

075

1

0 2 4 6 8 10

m+

t rect 1 t 2

0

05

1

15

2

0 2 4 6 8 10

|li| m

ax

t rect 1 t 2

m+ = 05m+ = 05



0

025

05

075

1

0 20 40 60 80

m+

t rect 1 t 2

0

1

2

3

4

0 20 40 60 80

|li| m

ax

t rec

t 1 t 2

m+ = 05m+ = 05m+ = 05

1

2

15 16 17

|li| m

ax

t rec




0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 005 010

002

004

006

008

01

b = 20 trec


= 30 msec

b = 20 trec

= 165 msec




0

100

200

300

400

500

05 1 15 2 25 3

Per

iod

(iter

atio

ns)

ct rect rec

b =100b = 50b = 10







0

5

10

15

20

0 2 4 6 8 10

t rec

b


Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase





6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References


































0

025

05

075

1

0 20 40 60 80

m+

t rect 1 t 2

0

1

2

3

4

0 20 40 60 80

|li| m

ax

t rec

t 1 t 2

m+ = 05m+ = 05m+ = 05

1

2

15 16 17

|li| m

ax

t rec




0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 005 010

002

004

006

008

01

b = 20 trec


= 30 msec

b = 20 trec

= 165 msec




0

100

200

300

400

500

05 1 15 2 25 3

Per

iod

(iter

atio

ns)

ct rect rec

b =100b = 50b = 10







0

5

10

15

20

0 2 4 6 8 10

t rec

b


Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase





6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References


































0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 05 10

02

04

06

08

1

0 005 010

002

004

006

008

01

b = 20 trec


= 30 msec

b = 20 trec

= 165 msec




0

100

200

300

400

500

05 1 15 2 25 3

Per

iod

(iter

atio

ns)

ct rect rec

b =100b = 50b = 10







0

5

10

15

20

0 2 4 6 8 10

t rec

b


Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase





6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References


































0

100

200

300

400

500

05 1 15 2 25 3

Per

iod

(iter

atio

ns)

ct rect rec

b =100b = 50b = 10







0

5

10

15

20

0 2 4 6 8 10

t rec

b


Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase





6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References


































0

5

10

15

20

0 2 4 6 8 10

t rec

b


Ferromagnetic phase

0

200

400

600

800

0 40 80 120 160 200

t rec

b

Paramagnetic phase

Oscillatory phase

Ferromagnetic phase





6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References


































6 Discussion













D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References









































D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References




































D D

0

BBB




iexclUmC 0


iexclU (1iexclmC )

1

CCCA (A1)






UxC 1 iexcl 1trec

iexcl UmC iexcl l 0





sup31 iexcl

1trec

iexcl U2

iexcl l

acute2 sup32b

c C 2iexcl l

acuteC

bUc C 2

sup31 iexcl

1trec

iexcl U2

iexcl l

acuteD 0 (A4)

which gives

l2 D 1 iexcl 1trec

iexcl U2

(A5)


1 iexcl 1trec

iexcl U2

iexcl l

acute sup32b

c C 2iexcl l

acuteC

bUc C 2

D 0 (A6)

Acknowledgments



References


































References













































Documents

Associative Memory with Dynamic Synapses