Path-integrals and large deviations in stochastic hybridsystems

Paul C Bressloff1

1Department of Mathematics, University of Utah

June 17, 2014

Part I. Path-integral representation of an SDE

LANGEVIN EQUATION WITH WEAK NOISE

Consider the scalar SDE

dX(t) = A(X)dt +√εdW(t),

for 0 ≤ t ≤ T and initial condition X(0) = x0. Here W(t) is a Wienerprocess and the noise is taken to be weak (ε 1).

Discretizing time by dividing the interval [0,T] into N equal subintervalsof size ∆t such that T = N∆t and setting Xn = X(n∆t), we have

Xn+1 − Xn = A(Xn)∆t +√ε∆Wn,

with n = 0, 1, . . . ,N − 1, ∆Wn = W((n + 1)∆t)−W(n∆t)

〈∆Wn〉 = 0, 〈∆Wm∆Wn〉 = ∆tδm,n.

Let X and W denote the vectors with components Xn and Wn

respectively.

CONDITIONAL PROBABILITY DENSITY

Conditional probability density function for X = x given a particularrealization w of the stochastic process W (and initial condition x0) is

P(x|w) =∏N−1

n=0 δ (xn+1 − xn − A(xn)∆t−√ε∆wn) .

Inserting the Fourier representation of the Dirac delta function,

δ(xm+1 − zm) =1

2π

∫ ∞−∞

e−ixm(xm+1−zm)dxm,

gives

P(x|w) =

N−1∏m=0

[∫ ∞−∞

e−ipm(xm+1 − xm − A(xm)∆t−√ε∆wm

) dpm

2π

].

The Gaussian random variable ∆Wn has the probability density function

P(∆wn) =1√

2π∆te−∆w2

n/2∆t.

JOINT PROBABILITY DENSITY

Setting

P(x) =

∫P[x|w]

N−1∏n=0

P(∆wn)d∆wn

and performing the integration with respect to ∆wn by completing thesquare, we obtain the result

P(x) =

N−1∏m=0

[∫ ∞−∞

e−ipm(xm+1−xm−A(xm)∆t)e−εp2m∆t/2 dpm

2π

].

Performing the Gaussian integration with respect to pm, we have

P(x) =

N−1∏m=0

1√2πε∆t

e−(xm+1−xm−A(xm)∆t)2/(2ε∆t)

= N exp

[− 1

2ε

N−1∑m=0

(xm+1 − xm

∆t− A(xm)

)2∆t

],

withN = 1(2πε∆t)N/2 .

ONSAGER-MACHLUP PATH INTEGRAL

Define expectations according to

E[F(X)] =

∫F(x)P(x)dx1 . . . xN

for any integrable function F.

Take the continuum limit ∆t→ 0,N →∞ with N∆t = T fixed. NowP[x] is a probability density functional over the different paths x(t)T

0realized by the original SDE with X(0) = x0:

P[x] ∼ exp[− 1

2ε

∫ T

0(x− A(x))2dt

],

The expectation of a functional F[x] is given by the Onsager-Machluppath integral

E[F[x]] =

∫F[x]P[x]D(x),

where D[x] is an appropriate measure.

VARIATIONAL PRINCIPLE

The conditional probability density that the stochastic process X(t)reaches a point x at time t = τ given that it started at x0 at time t = 0 is

P(x, τ |x0) =

∫ x(τ)=x

x(0)=x0

exp[− 1

2ε

∫ τ

0(x− A(x))2dt

]D[x].

In the limit ε→ 0, we can use the method of steepest descents to obtainthe approximation

P(x, τ |x0) ∼ exp[−Φ(x, τ |x0)

ε

],

where Φ is the quasipotential

Φ(x, τ |x0) = infx(0)=x0,x(τ)=x

S[x],

with

S[x] =

∫ τ

0L(x, x)dt, L(x, x) =

12

(x− A(x))2

VARIATIONAL PRINCIPLE II

Variational problem that minimizes the functional S[x] over trajectoriesfrom x(t)τ0 with x(0) = x0 and x(τ) = x (most probable path)

We can identify S[x] as a “classical action” with correspondingLagrangian L(x, x)

Most probable path is given by the solution to the Euler-Lagrangeequation

ddt∂L∂x

=∂L∂x.

Substituting for L yields

x = A(x)A′(x)

that is,

x(t)2 = A(x(t))2 + constant

STEADY-STATE DENSITY

Suppose that in the zero noise limit there is a globally attracting fixedpoint xs such that A(xs) = 0.

Approximation of steady-state density can be obtained by solving theEuler-Lagrange equation with x(−∞) = xs and x(τ) = x. This yieldsx = −A(x).

The quasipotential is

Φ(x, τ) = −2∫ τ

−∞A(x)xdt = 2

∫ τ

−∞U′(x)xdt = 2

∫ x

xs

U′(x)dx = 2U(x).

Hence, we obtain the expected result that the stationary density is

P(x) ∼ e−2U(x)/ε.

MULTI-VARIATE PATH-INTEGRAL

Consider the multivariate SDE

dXi(t) = Ai(X)dt +√ε∑

j

bij(X)dWi(t),

for i = 1, . . . , d with Wi(t) a set of independent Wiener processes.

Generalizing the path integral method to higher dimensions, one obtainsthe action functional

S[x] =12

∫ T

0

d∑i,j=1

(xi(t)− Ai(x(t)))D−1ij (xj(t)− Aj(x(t)))dt,

where D = bbtr is the diffusion matrix.

FOKKER-PLANCK EQUATION

Consider the FP equation corresponding to the scalar SDE:

∂p∂t

= −∂[A(x)p(x, t)]∂x

+ε

2∂2p(x, t)∂x2 ≡ −∂J(x, t)

∂x,

where

J(x, t) = − ε2∂p(x, t)∂x

+ A(x)p(x, t).

Suppose that the deterministic equation x = A(x) has a stable fixed pointat x−, A(x−) = 0, with 0 < x− < x0.

Impose an absorbing boundary condition at x0 and a reflectingboundary condition at x = 0:

p(x0, t) = 0, J(0, t) = 0

WKB APPROXIMATION

We seek a quasistationary solution of the WKB form

φε(x) ∼ K(x; ε)e−Φ(x)/ε,

with K(x; ε) ∼∑∞m=0 εmKm(x).

Substitute into the stationary FP equation and Taylor expand withrespect to ε.

Lowest order equation is

12

(∂Φ(x)

∂x

)2

+ A(x)∂Φ(x)

∂x= 0.

Similarly, collecting O(ε) terms yields the following equation for theleading contribution K0 to the pre factor:[

∂Φ

∂x+ A(x)

]∂K0

∂x= −

[A′(x) +

12∂2Φ(x)

∂x2

]K0(x).

HAMILTON-JACOBI EQUATION

Introducing the time-independent “Hamiltonian”

H(x, p) =p2

2+ A(x)p,

we can rewrite lowest order equation as

H(x,Φ′(x)) = 0.

Hamiltonian H describes a “particle” with position x and conjugatemomentum p evolving according to Hamilton’s equations

x =∂H∂p

= p + A(x), p = −∂H∂x

= −pA′(x).

Performing the Legendre transformation

H(x, p) = px− L(x, x), p =∂L∂x

we recover Lagrangian of Onsager-Machlup path integral:

L(x, x) =12

(x− A(x))2

Part II. Path-integral representation of a stochas-tic hybrid system

EXAMPLES OF STOCHASTIC HYBRID SYSTEMS

Stochastic neural populations(PCB/Newby 2013)

STOCHASTIC PROCESS

A process of change governed byprobabilities at each step.

2 | JANUARY 2002 | VOLUME 3 www.nature.com/reviews/neuro

R E V I E W S

advantage in overall predominance, as indexed by thepercentage of total viewing time for which it is domi-nant. So, for example, a high-contrast rival figure willbe visible for a greater percentage of time than a low-contrast one19, a brighter stimulus patch will predominateover a dimmer one20, moving contours will enjoy anadvantage over stationary ones21, and a densely contouredfigure will dominate a sparsely contoured one17,22. Doesa ‘strong’ rival figure enjoy enhanced predominancebecause its periods of dominance last longer, on average,than those of a weaker figure, or because its periods ofsuppression are abbreviated, on average? The evidencefavours the latter explanation: variations in the stimulusstrength of a rival target primarily alter the durations of suppression of that target, with little effect on itsdurations of dominance17,23.

Can these unpredictable fluctuations in dominanceand suppression be arrested by mental will power?Hermann von Helmholtz, among others, believed thatthey could24. Observing rivalry between sets of orthogo-nally oriented contours presented separately to the twoeyes, Helmholtz claimed to be able to hold one set ofcontours dominant for an extended period of time byattending vigorously to some aspect of those contours,such as their spacing. Ewald Hering, Helmholtz’s long-standing scientific adversary, characteristically disagreedwith this claim, arguing that any ability to deliberatelymaintain dominance of one eye’s view could be chalkedup to eye movements and differential retinal adapta-tion25. Which view does the weight of evidence favour? Itdoes appear that, with prolonged practice, attention canbe used to alter the temporal dynamics of rivalry26 with-out resorting to oculomotor tricks. However, this evi-dence also indicates that observers cannot maintaindominance of one rival figure to the exclusion ofanother26, even when that temporarily dominant figurecomprises interesting, potentially personal visual mater-ial27 — an attended rival figure eventually succumbs tosuppression despite concentrated efforts to maintain itsdominance. In this respect, binocular rivalry differsfrom dichotic listening, in which a listener can maintainfocused attention indefinitely on one of two competingmessages broadcast to the two ears.

There is reason to believe that ‘top–down’ atten-tional modulation of rivalry operates by boosting theeffective strength of a stimulus during dominance. Ooiand He28 found that a dominant stimulus was less sus-ceptible to a perturbing event presented to the othereye when observers voluntarily focused attention onthat dominant stimulus. However, we know that volun-tary attention cannot be guided by visual cues pre-sented during suppression phases of rivalry29; evidently,then, voluntary attention does not have access to infor-mation portrayed in a suppressed figure. However,involuntary attention can be captured during suppres-sion: stimulus events known to capture involuntaryattention — such as the sudden onset of motion in apreviously stationary figure — are sufficient to rescuea stimulus from suppression, thrusting it into con-scious awareness at the expense of its competitor30–32.So, voluntary, ‘endogenous’ attention seems to operate

Definitive answers to these questions are not yetavailable, but this review summarizes what we know atpresent. We start with an overview of the hallmark per-ceptual properties of binocular rivalry, for these will illu-minate the search for its neural concomitants. From theoutset, it is important to keep in mind that rivalry prob-ably does not stem from a single, omnibus process; inour view, it is near-sighted to speak of ‘the’ neural mech-anism of binocular rivalry. Instead, multiple neuraloperations are implicated in rivalry, including: registra-tion of incompatible visual messages arising from thetwo eyes; promotion of dominance of one coherent per-cept; suppression of incoherent image elements; andalternations in dominance over time. These distinctoperations might be implemented by neural events dis-tributed throughout the visual pathways, an overarchingtheme that we shall develop in this review.

Perceptual characteristics of rivalryTemporal dynamics. Fluctuations in dominance andsuppression during rivalry are not regular, like the oscil-lations of a pendulum. Instead, successive periods ofdominance of the left-eye stimulus and the right-eyestimulus are unpredictable in duration, as if being gener-ated by a STOCHASTIC PROCESS driven by an unstable timeconstant9,17,18. It is possible, however, to bias this dynamicprocess by boosting the strength of one rival figure overanother. In this case, the ‘stronger’ competitor enjoys an

a

b d

c

Figure 1 | Examples of some well-known ambiguousfigures, the perceptual appearance of which fluctuatesover time despite unchanging physical stimulation.a | The Necker cube. b | Rubin’s vase/face figure. c | E. G.Boring’s old lady/young woman figure. d | Monocular rivalry, inwhich two physically superimposed patterns that are dissimilarin colour and orientation compete for perceptualdominance113. Readers are encouraged to view each figure fordurations sufficient to experience alternations in perception,which, for naive viewers, can take some time. Evidently, whenone views figures such as these, the brain vacillates betweenalternative neural states; for this reason, such multistablefigures offer a promising means to study the neural bases ofvisual perception.

© 2001 Macmillan Magazines Ltd

gene networks (Newby 2012)

© 2005 Nature Publishing Group

Repressed promoter (R)

kon sA

sR

sP

δM δPkoff

mRNA (M) Protein (P)Active promoter (A)

d [M]

dt=

kon + koff

A

V+

kon + koff

R

V− M

[ ]on offk ks sMδ

d [P ]

dt= sP [M ] ]− P[Pδ

the random formation and decay of single molecules and multi-component complexes explicitly. As a result, the deterministic approach cannot capture the poten-tially significant effects of factors that cause stochasticity in gene expression.

In certain circumstances, deterministic simula-tions of the model in FIG. 1 predict intracellular protein concentrations that are similar to those predicted by stochastic simulations. The conditions that need to be satisfied for the predictions of the two approaches to be similar are large system size (high numbers of expressed mRNA and proteins, and large cell vol-umes) and fast promoter kinetics (BOX 1; see below).

These conditions are met in the example illustrated in FIG. 2a, in which the protein concentration (the overall measure of gene expression) predicted by a stochastic simulation fluctuates with very low amplitude around the average level predicted by a deterministic simula-tion. Correspondingly, the relative deviation from the average, measured by the ratio η of the standard deviation σ to the mean N, is quite small. This ratio η (or, alternatively, η2) is typically referred to as the coefficient of variation, or the noise.

When the conditions required for good agreement between deterministic and stochastic simulations are not fulfilled, the effects of molecular-level noise can

Figure 1 | A model of the expression of a single gene. Each step represents several biochemical reactions, which are associated with mRNA and protein production, transitions between promoter states and the decay of mRNA and protein. kon, koff, sA, sR, sP, δM and δP are the rate constants associated with these steps, as indicated. These reactions involve binding and dissociation events that occur at random at the molecular level. This is ignored in deterministic models of gene expression, which typically describe the different steps in terms of reaction rates. Stochastic models generally describe each step as a single random event, with a reaction time that shows an exponential distribution. All steps are assumed to obey first-order kinetics. The ratios sP /δM (the average number of proteins produced per mRNA) and sA/koff (the average number of mRNA produced between successive promoter activation and inactivation events) are referred to as the translational and transcriptional efficiency, respectively.

Box 1 | Deterministic rate equations and stochastic models of gene expression

Rate equations One mathematical framework for describing gene expression uses deterministic rate equations to calculate the concentrations of mRNA [M] and proteins [P]. For the model in FIG. 1, with a single gene copy, these equations are:

(1)

(2)

where V is the cell volume; the terms δM[M] and δP[P] are the degradation rates for mRNA and proteins, respectively; and the term sP[M] is the rate of protein synthesis. The rate constants kon and koff govern transitions between the active and repressed states of the promoter. Therefore, the ratios kon/(koff + kon) and koff /(kon + koff) in equation 1 are the fraction of time that the gene spends in the active and repressed states, respectively (that is, the promoter is assumed to be in chemical equilibrium). Consequently, mRNA production occurs at a constant rate, which is given by the weighted average of the activated synthesis (sA) and repressed synthesis (sR) mRNA synthesis rates.

The macroscopic limit and promoter kineticsThe above equations represent a valid approximation of the stochastic description when two limits are satisfied (FIG. 2a). The first is the macroscopic limit in which sR, sA and V become large, with the ratios sR/V and sA/V remaining constant. The second is the limit of fast chemical kinetics in which kon and koff become large, with their ratio remaining constant. Note that these limits do not alter equations 1 and 2. In FIG. 2b, the limit of fast chemical kinetics is satisfied, whereas fluctuations that are due to small system size are large (see main text). The reverse is true in FIGS 3a,b, where the number of expressed molecules is high, but the transitions between promoter states occur less frequently. Typically, concentration fluctuations scale in the form 1/√V for small system size effects (corresponding to 1/√N scaling, as [N] = N/V), and in the form 1/√koff + kon for slow chemical kinetic effects19.

452 | JUNE 2005 | VOLUME 6 www.nature.com/reviews/genetics

R E V I E W S

Motor-driven intracellular transport(PCB/Newby 2011),

lengths or velocities, as observed by mutating dynein on lipid-droplets (13, 14) and kinesin on axonal protein carrying vesicles(19). However, in melanophores, kinesin inactivation leads tobreakdown of plus motion and increased minus run lengths (11).

Interfering with the dynein–cofactor dynactin impairs trans-port in both directions in melanophores (16), but impairs minusand enhances plus transport of adenovirus particles (20). In theonly in vitro experiment concerning bidirectional transport (21),a motility assay of kinesin and dynein, it was observed thatincreasing the number of dyneins enhances minus and impairsplus end transport.

As shown here, all of these experimental observations areconsistent with the tug-of-war mechanism. In fact, we present anexplicit tug-of-war model that takes into account the experi-mentally known single motor properties and makes quantitativepredictions for bidirectional transport. In our model, the motorsact independently and interact only mechanically via theircommon cargo. We find seven possible motility regimes for cargotransport. Three of these regimes are dominated by the threeconfigurations (0), (), and () in Fig. 1 and represent nomotion, fast plus motion, and fast minus motion of the cargo,respectively. The other motility states are combinations thereof;in particular, there are the two regimes, () and (0), wherethe cargo displays fast bidirectional transport without and withpauses, respectively. During fast plus or minus motion, only onemotor type is pulling most of the time and the tug-of-war appearsto be coordinated.

The different motility regimes are found for certain ranges ofsingle-motor parameters such as stall force and MT affinity.Small changes in these parameters lead to drastic changes incargo transport, e.g., from fast plus motion to bidirectionalmotion or no motion. We propose that cells could use thesensitivity of the transport to the single-motor properties toregulate its traffic in a very efficient manner. We illustrate thisgeneral proposal by providing an explicit and quantitative tug-of-war model for the lipid-droplet system.

ResultsModel. To study the bidirectional transport of cargos, we devel-oped a model for a cargo to which N plus and N minus motorsare attached. Typically these numbers will be in the range of 1to 10 motors as observed for many cargos in vivo (12, 22, 23). ForN 0 or N 0, we recover the model for cooperativetransport by a single motor species as studied in ref. 24. Wecharacterize each motor species by six parameters as measuredin single molecule experiments [see Table 1 and supportinginformation (SI) Text] as follows: it binds to a MT with thebinding rate 0 and unbinds with the unbinding rate 0, whichincreases exponentially under external force, with the force scalegiven by the detachment force Fd. When bound to the MT, themotor walks forward with the velocity vF, which decreases withexternal force and reaches zero at the stall force Fs. Undersuperstall external forces, the motor walks backward slowly withbackward velocity vB.

The motors on the cargo bind to and unbind from a MT in a

stochastic fashion, so that the cargo is pulled by n N plusand n N minus motors, where n and n f luctuate with time(see Fig. 2). We have derived the rates for unbinding of one ofthe bound motors and for binding of an additional motor on thecargo from the single motor rates under the assumption that: (i)the presence of opposing motors induces a load force, and (ii)this load force is shared equally by the bound motors belongingto the same species (see SI Text). We obtain a Master equationfor the motor number probability p(n, n) that the cargo ispulled by n plus and n minus motors. The observable cargomotion is characterized by the motor states (n, n) with highprobability. If there is high probability for a state (n, 0) or (0,n) with only one motor species bound, corresponding to Fig.1() and (), the cargo exhibits fast plus or minus motion,respectively. If there is high probability for a state with bothmotor species active, i.e., n 0 and n 0, the cargo displaysonly negligible motion into the direction of the motors that ‘‘win’’the tug-of-war, because the losing motors walk backward onlyvery slowly. This corresponds to the blockade situation depictedin Fig. 1 (0).

Motility States for the Symmetric Case. We first studied the instruc-tive symmetric case, for which the number of plus and minus motorsare the same and where plus and minus motors have identicalsingle-motor parameters except for their preferred direction ofmotion. Apart from being theoretically appealing, this symmetricsituation can be realized in vitro if cargos are transported by a singlemotor species along antiparallel MT bundles, and can also be usedin vivo provided plus and minus end transport exhibit sufficientlysimilar transport characteristics.

We solved our model for fixed motor numbers N N andfixed single-motor parameters and determined the probabilitydistribution p(n, n) (see SI Text). Depending on the values of

(0) (−)

− − − ++ +

(+)

Fig. 1. Cargo transport by 2 plus (blue) and 2 minus (yellow) motors: possibleconfigurations (0), (), and () of motors bound to the MT. For configuration(0), the motors block each other so that the cargo does not move. Forconfiguration () and (), the cargo exhibits fast plus and minus motion,respectively.

Table 1. Values of the single-motor parameters for kinesin 1,cytoplasmic dynein, and an unknown plus motor (kin?) thattransports Drosophila lipid droplets

Parameter Kinesin 1 Dynein kin?

Stall force Fs, pN 6 (29, 30) 1.1* (12, 27) 7 (31) 1.1* (12)Detachment force Fd, pN 3 (30) 0.75* 0.82*Unbinding rate 0, s1 1 (30, 32) 0.27* (27, 33) 0.26*Binding rate 0, s1 5 (34) 1.6* (33, 35) 1.6*Forward velocity vF, m/s 1 (32, 36) 0.65* (33, 37) 0.55*Back velocity vB, nm/s 6 (36) 72* 67*

The kinesin 1 values have been taken from the cited references. The starredvalues are obtained by fitting experimental data of Drosophila lipid-droplettransport and are consistent with the cited references.

Fig. 2. A cargo with N 3 plus (blue) motors and N 2 minus (yellow)motors is pulled by a fluctuating number of motors bound to the MT. Theconfiguration in the middle corresponds to (n, n) (2, 1). Only five of 12possible (n, n) configurations are displayed.

4610 www.pnas.orgcgidoi10.1073pnas.0706825105 Muller et al.

Dendritic NMDA spikes (PCB/Newby2014)

thick apical

branch

thin apical

tufts

thin basal

branches

(a)

axon

thin oblique

tufts

(b)

NMDA spike

plateau potentialNa+ spikelet

subthreshold EPSP20 mV

50 ms

1D STOCHASTIC HYBRID SYSTEM

Consider the 1D system

dxdt

=1τx

Fn(x), x ∈ R, n = 0, . . . ,K − 1

Jump Markov process n′ → n with transition rates Wnn′(x)/τn.

Set τx = 1 and introduce the small parameter ε = τn/τx

CK equation is

∂p∂t

= −∂[Fn(x)pn(x, t)]∂x

+1ε

K−1∑n′=0

Ann′(x)pn′(x, t)

where

Ann′(x) = Wnn′(x)−K−1∑m=0

Wmn(x)δn′,n.

In the limit ε→ 0, obtain mean-field equation

dxdt

= F(x) ≡K−1∑n=0

Fn(x)ρn(x),

where∑

m∈I Anm(x)ρm(x) = 0.

PATH-INTEGRAL I

Discretize time by dividing a given interval [0,T] into N equalsubintervals of size ∆t such that T = N∆t and set

xj = x(j∆t), nj = n(j∆t)

The conditional probability density for x1, . . . , xN given x0 and a par-ticular realization of the stochastic discrete variables nj, j = 0, . . . ,N−1, is

P(x1, . . . , xN|x0, n0, . . . , nN−1) =

N−1∏j=0

δ(

xj+1 − xj − Fnj (xj)∆t)

Using the Fourier representation of the Dirac delta function,

P(x1, . . . , xN|x0, n0,n) =

N−1∏j=0

[∫ ∞−∞

e−ipj

(xj+1 − xj − Fnj (xj)∆t

)dpj

2π

]

≡N−1∏j=0

[∫ ∞−∞

Hnj (xj+1, xj, pj)dpj

2π

]

PATH-INTEGRAL II

On averaging with respect to the intermediate states n = (nq, . . . , nK−1),we have

P(x1, . . . , xN|x0, n0) =

N−1∏j=0

∫ ∞−∞

dpj

2π

∑n1,...,nN−1

N−1∏j=0

Tnj+1,nj (xj)Hnj (xj+1, xj, pj)

where

Tnj+1,nj (xj) ∼ Anj+1,nj (xj)∆tε

+ δnj+1,nj

(1−

∑m

Am,nj (xj)∆tε

)+ o(∆t)

=

(δnj+1,nj + Anj+1,nj (xj)

∆tε

).

PATH-INTEGRAL III

Consider the eigenvalue equation

∑m

[Anm(x) + qδn,mFm(x)] R(s)m (x, q) = λs(x, q)R(s)

n (x, q),

and let ξ(s)m be the adjoint eigenvector.

Insert multiple copies of the identity∑s

ξ(s)m (x, q)R(s)

n (x, q) = δm,n

into the discrtetized path-integral with (x, q) = (xj, qj) at the jth time-step

PATH-INTEGRAL IV

Find that

P(xN, nN|x0, n0) ≡N−1∏j=1

∫ ∞−∞

dxjP(x1, . . . , xN, nN|x0, n0)

=

N−1∏j=1

∫ ∞−∞

∫ ∞−∞

dxjdpj

2π

∑n1,...,nN−1

∑s0,...,sN−1

N−1∏j=0

R(sj)nj+1 (xj, qj)ξ

(sj)nj (xj, qj)

exp

∑j

[λsj (xj, qj)− iεpj

xj+1 − xj

∆t

]∆tε

exp(

[iεpjFnj (xj)− qjFnj (xj)]∆tε

).

Discretized path integral is independent of the qj. Set qj = iεpj for all jand eliminate the final exponential factor.

Sum over the intermediate discrete states nj using the orthogonalityrelation ∑

n

R(s)n (x, q)ξ

(s′)n (x, q) = δs,s′ .

PATH-INTEGRAL V (PCB AND NEWBY 2014)

Perron-Frobenius theorem shows that there exists a real, simple Perroneigenvalue labeled by s = 0, say, such that λ0 > Re(λs) for all s > 0

Hence, set sj = 0 and take the continuum limit to obtain the followingpath-integral from x(0) = x0 to x(τ) = x (after performing the change ofvariables iεpj → pj (complex contour deformation):

P(x, n, τ |x0, n0, 0) =

x(τ)=x∫x(0)=x0

exp(−1ε

∫ τ

0[px− λ0(x, p)]dt

)D[p]D[x]

Dropped factor R(s)0 (x, p(τ))ξ

(0)n0 (x0, p(0))

VARIATIONAL PRINCIPLE

Applying steepest descents to path integral yields a variational principlein which optimal paths minimize the action

S[x, p] =

∫ τ

0[px− λ0(x, p)] dt.

Hence, we can identify the Perron eigenvalue λ0(x, p) as a Hamiltonianand the optimal paths are solutions to Hamilton’s equations

x =∂H∂p

, p = −∂H∂x

, H(x, p) = λ0(x, p)

Deterministic mean field equations and optimal paths of escape from ametastable state both correspond to zero energy solutions.

Setting λ0 = 0 in eigenvalue equation gives

∑m

[Anm(x) + pδn,mFm(x)] R(0)m (x, p) = 0

“ZERO ENERGY” PATHS

Ω

∂Ω

separatrix

xs

Ω

∂Ω

xs

a b

(a) Deterministic trajectories converging to a stable fixed point xS.Boundary of basin of attraction formed by a union of separatrices

(b) Noise-induced paths of escape

MEAN-FIELD EQUATIONS

We have the trivial solution p = 0 and R(0)m (x, 0) = ρm(x) with

∑m

Anm(x)ρm(x) = 0

Differentiating the eigenvalue equation with respect to p and thensetting p = 0, λ0 = 0 shows that

∂λ0(x, p)

∂p

∣∣∣∣p=0

ρn(x) = Fn(x)ρn(x) +∑

m

Anm(x)∂R(0)

m (x, p)

∂p

∣∣∣∣∣p=0

Summing both sides wrt n and using∑

n Anm = 0,

∂λ0(x)

∂p

∣∣∣∣p=0

=∑

n

Fn(x)ρn(x)

Hamilton’s equation x = ∂λ0(x, p)/∂p recovers mean-field equation

x =∑

n

Fn(x)ρn(x).

MAXIMUM-LIKELIHOOD PATHS OF ESCAPE

Unique non-trivial solution p = µ(x) with positive eigenvectorR(0)

m (x, µ(x)) = ψm(x):

∑m

[Anm(x) + µ(x)δn,mFm(x)]ψm(x) = 0

Recovers leading order equation for WKB quasipotential Φ(x) withΦ′(x) = µ(x) and

S[x, p] ≡∫ τ

−∞[px− λ0(x, p)] dt =

∫ x

xs

Φ′(x)dx.

Part III. Stochastic ion-channels revisited

STOCHASTIC MORRIS-LECAR MODEL

Let n, n = 0, . . . ,N be the number of open sodium channels:

dvdt

= Fn(v) ≡ 1N

f (v)n− g(v),

with f (v) = gNa(VNa − v) and g(v) = −geff[Veff − v] + Iext.

The opening and closing of the ion channels is described by abirth-death process according to

n→ n± 1,

with rates

ω+(n) = α(v)(N − n), ω−(n) = βn

Take

α(v) = β exp(

2(v− v1)

v2

)for constants β, v1, v2.

CHAPMAN-KOLMOGOROV EQUATION I

Introduce the joint probability density

Probv(t) ∈ (v, v + dv), n(t) = n = pn(v, t)dv,

for given initial data

Differential Chapman-Kolmogorov (CK) equation (dropping theexplicit dependence on initial conditions)

∂pn

∂t= −∂[Fn(v)pn(v, t)]

∂v

+1ε

[ω+(n− 1)pn−1(v, t) + ω−(n + 1)pn+1(v, t)− (ω+(n) + ω−(n))pn(v, t)]

Introduced small parameter ε - opening and closing of sodium channelsmuch faster than relaxation dynamics of voltage

CHAPMAN-KOLMOGOROV EQUATION II

Rewrite CK equation in the more compact form

∂pn

∂t= −∂[Fn(v)pn(v, t)]

∂v+

1ε

∑n′

Anm(v)pm(v, t),

An,n−1 = ω+(n− 1), Ann = −ω+(n)− ω−(n), An,n+1 = ω−(n + 1).

There exists a unique steady state density ρn(v) for which∑m

Anm(v)ρm(v) = 0

where

ρn(v) =N!

(N − n)!n!a(v)nb(v)N−n, a(v) =

α(v)

α(v) + β, b(v) = 1− a(v).

MEAN-FIELD LIMIT

In the limit ε→ 0, we obtain the mean-field equation

dvdt

=∑

n

Fn(v)ρn(v) = a(v)f (v)− g(v) ≡ −dΨ

dv,

Assume deterministic system operates in a bistable regime

!(v)

v [mV]

v- v*

v+

-100 -80 -60 -40 -20 0 20 40 60 80 100

Iext = I*

Iext < I*

PERRON EIGENVALUE

Eigenvalue equation for λ0 and R(0) = ψ:

(N − n + 1)αψn−1 − [λ0 + nβ + (N − n)α]ψn + (n + 1)βψn+1

= −p( n

Nf − g

)ψn

Consider the trial solution

ψn(x, p) =Λ(x, p)n

(N − n)!n!,

Yields the following equation relating Λ and µ:

nαΛ

+ Λβ(N − n)− λ0 − nβ − (N − n)α = −p( n

Nf − g

).

Collecting terms independent of n and terms linear in n yields

p = − Nf (x)

(1

Λ(x, p)+ 1)

(α(x)− β(x)Λ(x, p)) ,

and

λ0(x, p) = −N(α(x)− Λ(x, p)β(x))− pg(x).

PERRON EIGENVALUE II

Eliminating Λ from these equation gives

p =1

f (x)

(Nβ(x)

λ0(x, p) + Nα(x) + pg(x)+ 1)

(λ0(x, p) + pg(x))

Obtain a quadratic equation for λ0:

λ20 + σ(x)λ0 − h(x, p) = 0.

with

σ(x) = (2g(x)− f (x)) + N(α(x) + β(x)),

h(x, p) = p[−Nβ(x)g(x) + (Nα(x) + pg(x))(f (x)− g(x))].

The “zero energy” solutions imply that h(x, p) = 0

RECOVERS WKB QUASIPOTENTIAL

Non-trivial solution recovers result of WKB analysis

p = µ(x) ≡ Nα(x)f (x)− (α(x) + β)g(x)

g(x)(f (x)− g(x)).

The corresponding quasipotential Φ is given by

Φ(x) =

∫ x

µ(y)dy.

Analogous result in full ML model

Part IV. Higher-dimensional systems

D-DIMENSIONAL STOCHASTIC HYBRID SYSTEM

Consider the system

dxi

dt=

1τx

F(i)n (x), x ∈ RD, i = 1, . . . ,D

Jump Markov process n′ → n with transition rates Wnn′(x)/τn.

Set τx = 1 and introduce the small parameter ε = τn/τx

CK equation is

∂pn

∂t= −

∑i

∂[F(i)n (x)pn(x, t)]

∂xi+

1ε

∑n′

Ann′(x)pn′(x, t)

Ann′(x) = Wnn′(x)−∑

m

Wmn(x)δn′,n.

In the limit ε→ 0, obtain mean-field equation

dxi

dt= Fi(x) ≡

∑n

F(i)n (x)ρn(x),

where∑

m∈I Anm(x)ρm(x) = 0.

PATH-INTEGRAL

Proceeding as in the 1D case find that

pn(x, τ |x0,n0, 0) =

x(τ)=x∫x(0)=x0

D[p]D[x] exp(−1ε

S[x,p]

)

×R(0)n (x,p(τ))ξ

(0)n0 (x0,p(0))

with action

S[x,p] =

∫ τ

0

[D∑

i=1

pixi − λ0(x,p)

]dt.

Here λ0 is the Perron eigenvalue of the following linear operator equation

∑m

[Anm(x)Rm

(0)(x,p) + δn,m

D∑i=1

piF(i)m (x)

]R(0)

m (x,p) = λ0(x,p)R(0)n (x,p),

and ξ(0) is the corresponding adjoint eigenvector.

STOCHASTIC MORRIS-LECAR MODEL REVISITED

Take n ≤ N open Na+ channels and m ≤ M open K+ channels:

dvdt

= F(v,m, n) ≡ nN

fNa(v) +mM

fK(v)− g(v).

Each channel satisfies the kinetic scheme

Cαi(v)−→←−βi(v)

O, i = Na, K,

The Na+ channels fast relative to voltage and K+ dynamics.

Chapman–Kolmogorov (CK) equation,

∂p∂t

= −∂(Fp)

∂v+ LKp + LNap.

The jump operators Lj, j = Na,K, are defined according to

Lj = (E+n − 1)ω+

j (n) + (E−n − 1)ω−j (n),

with E±n f (n) = f (n± 1), ω−j (n) = nβj and ω+j (n) = (N − n)αj(v).

SMALL NOISE LIMIT

Introduce a small parameter ε 1 such that (in dimensionless units)

β−1Na = ε, M−1 = λMε,

Set w = m/M and write (m± 1)/M = w±M−1

Perturbation expansion in ε combines a system size expansion with aslow/fast analysis

We would like to determine the most probable or optimal paths ofescape from the resting state in the (v,w)-plane for small ε

For chemical master equations, the quasipotential of the WKBapproximation satisfies a Hamilton-Jacobi equation - the optimal pathsgiven by solutions to an effective Hamiltonian dynamical system

There is an underlying variational principle derived using largedeviation theory or path-integrals

WKB APPROXIMATION

Introduce quasistationary solution of the form

ϕ(v,w, n) = Rn(v,w) exp(−1ε

Φ(v,w)

),

where Φ(v,w) is the quasipotential

To leading order,

[LNa + pv + h(v,w, pw)] Rn(v,w) = 0,

where

pv =∂Φ

∂v, pw =

∂Φ

∂wand

h(v,w, pw) =βK

MλM

[(e−λMpw − 1)ω+

K (Mw, v) + (eλMpw − 1)ω−K (Mw, v)]

HAMILTON-JACOBI EQUATION

Introducing the ansatz

Rn(v,w) =Λ(v,w)n

(N − n)!n!,

yields a Hamilton-Jacobi equation for Φ:

0 = H(v,w, pw, pv) ≡ (a(v)fNa(v) + g(v))pv + h(v,w, pw)

− b(v)

N

[((2g(v) + fNa(v))pvh(v,w, pw) + (fNa(v) + g(v))g(v)p2

v + h(v,w, pw)2)

Solve for Φ using method of characteristics. Satisfy Hamilton’s equations

x = ∇pH(x,p), p = −∇xH(x,p).

for x = (v,w) and p = (pv, pw)

Interpret Φ(t) as the action with Φ(t) = p(t) · x(t), is a strictly increasingfunction of t, and the quasipotential is given by Φ(v,w) = Φ(t) at thepoint (v,w) = x(t).

SOLUTIONS OF HJ EQUATION (NEWBY,PCB,KEENER 2013) 4

C

VN

WNS

S

BN

BN

CP

CP

FIG. 2. Orange curves are SAP trajectories, shown until theyreach the metastable separatrix (S). The dashed red curveis a SAP that reaches S near the bottleneck (BN). All ofthe SAP trajectories that enter the shaded region are visu-ally indistinguishable from the dashed red line before crossingS. Deterministic trajectories are shown as black streamlines.The upper left inset is a close up of the caustic formationpoint (CP) with overlapping metastable trajectories. Levelcurves of Φ are shown inside the potential well region withgrey lines. Also shown are the caustic (C), v nullcline (VN),and w nullcline (WN). Parameter values are N = M = 40and λM = 0.25.

at the fixed point as a single trajectory and then fan outjust before reaching the metastable separatrix (Fig. 2).After crossing the separatrix, all of the SAP trajecto-ries eventually reach the caustic. Although all SAPs areequally likely to reach the separatrix, their likelihood ofreaching the caustic depends on their amplitude. Largeamplitude SAPs are less likely and reach the caustic farfrom the caustic formation point. Strictly speaking, themost probable SAP strikes the caustic formation point,but Φ increases by a very small amount in the shaded re-gion of Fig. 2 because SAP trajectories are very close todeterministic trajectories (black streamlines). (The rela-tive difference is |∆Φ| /Φc ≈ 0.01.) Hence, the stationarydensity (7) is nearly constant in the shaded region.

SAPs that cover the shaded region cross a very smallsegment of the separatrix, the center of which acts asa bottleneck for SAPs. The shaded region representsthe most likely, experimentally observable SAP trajecto-ries; it excludes the small amplitude SAPs that (crossingabove the bottleneck) strike very close to the caustic for-mation point and the far less probable SAPs that (cross-ing below the bottleneck) strike the caustic above or be-hind the potential well region. The portion of the SAPtrajectory between the fixed point and the bottleneck(see Fig. 2 dashed curve) represents the initiation phase;it is not constant and remains below the voltage nullcline.This behavior is confirmed by Monte-Carlo simulations(see supplementary material for details).

To summarize our results, we find that fluctuations inthe slow recovery dynamics of K+ channels significantly

affect spontaneous activity in the ML model. The max-imum likelihood trajectory during initiation of a SAPcan be thought of as a path of least resistance, drop-ping below the voltage nullcline where voltage increasesdeterministically. Hence, SAP initiation is more likelyto occur using the second of the two mechanisms men-tioned in the introduction: a burst of simultaneously-closing K+ channels causes v to increase. If one takesw to be constant, only the first mechanism is availableand the path is artificially constrained, which alters thequasipotential. In other words, constraining the path al-ters the effective energy barrier for SAP initiation, whichsignificantly affects determination of the spontaneous fir-ing rate. Although it is more difficult to construct anexit time problem in an excitable system, this can nowbe done using the metastable separatrix. The methodsused here are general and may lead to future studies ofnoise-induced dynamics in other nonlinear stochastic sys-tems. In particular, it would be interesting to extend thecurrent analysis to the Hodgkin-Huxley model, where theNa+ channels have a slow inactivating component.

∗ newby.23@mbi.osu.edu[1] J. White, J. Rubinstein, and A. Kay, Trends Neurosci.

23, 131 (2000).[2] E. Schneidman, B. Freedman, and I. Segev, Neural Com-

put. 10, 1679 (1998).[3] J. P. Keener and J. Sneyd, Mathematical physiology

(Springer, 2009).[4] vNa = 120mV, gNa = 4.4mS/cm2, vK = −84mV,

gK = 8mS/cm2, vleak = −60mV, gleak = 2mS/cm2,Cm = 20µF/cm2, βK = 0.02ms−1, Iapp = 0.06Cmveff ,ϕ = −0.1, veff = 52.8mV, γNa = 1.22/veff , κNa =−1.188 + 1.22veff , γK = 0.8/veff , κK = 0.8 + 0.8veff ,τm = 10ms.

[5] C. Morris and H. Lecar, Biophys. J. 35, 193 (1981).[6] C. W. Gardiner, Handbook of stochastic methods for

physics, chemistry, and the natural sciences (Springer-Verlag, 1983).

[7] I. A. Khovanov, A. V. Polovinkin, D. G. Luchinsky, andP. V. E. McClintock, Phys. Rev. E 87, 032116 (2013).

[8] J. P. Keener and J. M. Newby, Phys. Rev. E 84, 011918(2011).

[9] B. Lindner and L. Schimansky-Geier, Phys. Rev. E 60,7270 (1999).

[10] J. Grasman and J. Roerdink, Journal of StatisticalPhysics 54, 949 (1989).

[11] C. C. Chow and J. A. White, Biophys. J. 71, 3013 (1996).[12] J. M. Newby and J. P. Keener, Multiscale Model. Simul.

9, 735 (2011).[13] J. M. Newby, Physical Biology 9, 026002 (2012).[14] M. I. Dykman, D. G. Luchinsky, P. V. E. McClintock,

and V. N. Smelyanskiy, Phys. Rev. Lett. 77, 5229 (1996).[15] R. S. Maier and D. L. Stein, SIAM J. Appl. Math. 57,

752 (1997).[16] Z. Schuss, Theory and applications of stochastic pro-

cesses: an analytical approach (Springer, 2010).

Caustic (C), v nullcline (VN), and w nullcline (WN), metastableseparatrix (S), bottleneck (BN), caustic formation point (CP)

PATH-INTEGRAL FOR STOCHASTIC ML

Path-integral action is

S[x,w, px, pw] =

∫ τ

0[pxx + pww− λ0(x,w, px, pw)]dt

where λ0 is the Perron eigenvalue of the following linear operatorequation

λ0Rn = [LNa + Fn(x,w)px + h(x,w, pw)] R(0)n .

with Fn(x,w) = F(x,Mw, n)

The path-integral representation of the stationary density is then

pn(x,w, τ |x0,w0, 0) =

x(τ)=x∫∫x(0)=x0

exp(−1ε

S[x,w, px, pw]

)

R(0)n (x,p(τ))ξ

(s)n0 (x0,p(0))D[p]D[x]

PERRON EIGENVALUE DIFFERS FROM WKB HAMILTONIAN

Introduce the ansatz

R(0)n (v,w) =

Λ(v,w)n

(N − n)!n!

into eigenvalue equation.

Collecting terms linear in n gives

A(x,w) = αNa(x)

− 1N

(pxg(x,w) + h(x,w, pw)− λ0(x,w, px, pw)),

Collecting terms independent of nand substituting for A(x,w) gives the following quadratic equation for λ0:

λ20 − (2h(x,w, pw) + σ(x,w, px))λ0 +H(x,w, px, pw) = 0,

with

σ(x,w, px) = (2g(x) + f (x))px −N/(1− w∞(x))

andH the WKB Hamiltonian.

REFERENCES

1 PCB. Mean field theory and effects of fluctuations in stochastic hybridneural networks. (2014)

2 PCB and J. M. Newby. Path-integrals and large deviations in stochastichybrid systems. Phys. Rev. E (2014)

3 PCB and J. M. Newby. Stochastic hybrid model of spontaneous dendriticNMDA spikes. Phys. Biol. 11 016006 (2014).

4 J. M. Newby, PCB and J. P. Keeener. The effect of Potassium channels onspontaneous action potential initiation by stochastic ion channels. Phys.Rev. Lett. 111 128101 (2013).

5 PCB and J. M. Newby. Metastability in a stochastic neural networkmodeled as a jump velocity Markov process SIAM J. Appl. Dyn. Syst. 121394-1435 (2013).