Stochastic Simulation of Enzyme-Catalyzed Reactionswith Disparate Timescales

Debashis Barik,*yz Mark R. Paul,* William T. Baumann,y Yang Cao,§ and John J. Tysonz

*Department of Mechanical Engineering, yDepartment of Electrical and Computer Engineering, zDepartment of Biological Sciences,and §Department of Computer Science, Virginia Polytechnic Institute and State University, Blacksburg, Virginia

ABSTRACT Many physiological characteristics of living cells are regulated by protein interaction networks. Because the totalnumbers of these protein species can be small, molecular noise can have significant effects on the dynamical properties of aregulatory network. Computing these stochastic effects is made difficult by the large timescale separations typical of proteininteractions (e.g., complex formation may occur in fractions of a second, whereas catalytic conversions may take minutes). Exactstochastic simulation may be very inefficient under these circumstances, and methods for speeding up the simulation withoutsacrificing accuracy have been widely studied. We show that the ‘‘total quasi-steady-state approximation’’ for enzyme-catalyzedreactions provides a useful framework for efficient and accurate stochastic simulations. Themethod is applied to three examples: asimple enzyme-catalyzed reaction where enzyme and substrate have comparable abundances, a Goldbeter-Koshland switch,where a kinase and phosphatase regulate the phosphorylation state of a common substrate, and coupled Goldbeter-Koshlandswitches that exhibit bistability. Simulations based on the total quasi-steady-state approximation accurately capture the steady-state probability distributions of all components of these reaction networks. In many respects, the approximation also faithfullyreproduces time-dependent aspects of the fluctuations. The method is accurate even under conditions of poor timescaleseparation.

INTRODUCTION

Stochastic simulation is important for the study of chemicalreaction systems where the number of molecules of somespecies can become small. The exact stochastic simulationalgorithm of Gillespie (1,2) is the standard approach to sim-ulation in this setting, but can become too slow to be useful incases where fast reactions dominate the calculations but slowreactions determine the interesting evolution of the overallsystem. This may be the case, for example, in enzyme-cata-lyzed reactions where binding and unbinding of the enzymeand substrate often occur much more frequently than theproduct-producing reaction. Gillespie’s algorithm will spendmost of its time tracking binding and unbinding reactions thatessentially cancel each other out and produce no net conver-sion of substrate into product.Many approximations to Gillespie’s algorithm have been

proposed to reduce the bottleneck that occurs in systems withdisparate timescales. Tau-leaping (3–6) uses Poisson randomvariables to allow multiple firings of reaction channels pertime step. This approach can speed up simulations, but theexplicit version has difficulty with the stiff systems we con-sider. The implicit version (5) was proposed to solve stiffsystems; however, practical t-selection formulas were onlyproposed (6) for the explicit version. A t-selection formulabased on reversible reactions was proposed, but a formula for ageneral stiff system is not yet available. Other approaches rely

on separating the system into slow and fast reactions or species.The slow reactions can be simulatedwithGillespie’s algorithmand the fast reactions can behandled in amore efficientmanner.One approach is to simulate fast reactions using chemicalLangevin equations (7,8). Other techniques avoid simulat-ing the fast reactions altogether, instead using their averagevalues to derive the firing propensities of the slow reactions.The average values are obtained using the quasi-steady-stateapproximation (QSSA) (9) or the quasi-equilibrium approx-imation (also known as the partial equilibrium assumption)(10–12).In this article, we apply an idea, called the total QSSA

(TQSSA), to provide an algorithmic, fast, and accurate way tostochastically simulate enzyme-catalyzed reactions. TheTQSSA was developed in a deterministic setting to removethe restrictions associated with the standard QSSA applied toenzyme-catalyzed reactions (13). The TQSSA accuratelytracks intermediate complexes and can be extended straight-forwardly to complex systems of interacting enzymes (14).In the stochastic setting considered here, TQSSA allowsGillespie’s algorithm to be used to simulate the slow reactionswhile analytically accounting for the abundances of the fastspecies, as in the methods mentioned above. Thus, it avoidsthe bottleneck associated with simulating the fast reactionsdirectly. The main contribution of this article is to show thatstochastic TQSSA generalizes the QSSA results of Rao andArkin (9) and can improve the accuracy of the partial equi-librium assumption employed by others (10–12) by allowingthe slow timescales to affect the quasi-steady-state of the fastcomplexes. We provide a general framework for applyingstochastic TQSSA to chemical reaction networks and illus-

doi: 10.1529/biophysj.108.129155

Submitted January 10, 2008, and accepted for publication June 17, 2008.

Address reprint requests to John J. Tyson, Dept. of Biological Sciences,Virginia Polytechnic Institute, Blacksburg, VA 24061. Tel.: 540-231-4662;

Fax: 540-231-9307; E-mail: tyson@vt.edu.

Editor: Arthur Sherman.

! 2008 by the Biophysical Society0006-3495/08/10/3563/12 $2.00

Biophysical Journal Volume 95 October 2008 3563–3574 3563

trate its usefulness on a complex network exhibiting alterna-tive stable steady states. Our results show that for cases havingreasonable timescale separation between fast and slow reac-tions, the stochastic TQSSA simulations are nearly identicalto direct simulation of all the reactions using Gillespie’s sto-chastic simulation algorithm.

MICHAELIS-MENTEN KINETICS

Consider a simple chemical reaction where a substrate (S) isconverted irreversibly to a product (P) by an enzyme (E):

E1 S!k1

k!1

E :S/k2

P1E: (1)

In these reactions, k1 and k!1 are the rate constants forformation and dissociation of the enzyme-substrate complex(E : S), and k2 is the rate constant for the enzymatic conversionof S to P.With square brackets denoting the concentration of achemical species, the rate equations corresponding to theabove elementary reactions can be written as

d"S#dt

$ !k1"S#%ET ! "E:S#&1 k!1"E:S#; (2)

d"E:S#dt

$ !%k!1 1 k2&"E:S#1 k1"S#%ET ! "E:S#&; (3)

where ET $ "E#1"E :S# is the total enzyme concentration.We always use subscript T to indicate the total amount of achemical moiety. If the concentration of substrate is verymuch greater than ET; then the change in concentration of theenzyme-substrate complex (Eq. 3) will be fast in comparisonto the change in concentration of the free substrate. Underthese conditions, the enzyme-substrate complex rapidlyreaches its stationary state, i.e.,

d"E:S#dt

$ !%k!1 1 k2&"E:S#1 k1"S#%ET ! "E:S#& $ 0;

"E:S# $ ET"S#Km 1 "S#; (4)

where Km $ (k!1 1 k2)/k1 is called the Michaelis constant ofthe enzyme. Equation 4 is the QSSA for the enzyme-substratecomplex. In this case, the conversion of substrate into productis described by the well known Michaelis-Menten rate law:

d"S#dt

$ !k2"E:S# $ ! k2ET"S#Km 1 "S#: (5)

The condition for validity of the QSSA is ET ' S01Km;where S0 is the initial substrate concentration.When this condition is not satisfied, then both [S] and [E:S]

change on the same timescale. However, when k2 ' k!1, the‘‘total’’ substrate concentration, "S# $ "S#1"E:S#; changeson a slower timescale than the enzyme-substrate complex (13):

d"S#dt

$ !k2"E:S#; (6)

d"E:S#dt

$!%k!11k2&"E:S#1k1 S! "

!"E:S## $

ET!"E:S#% &:

(7)

Applying the QSSA, d[E:S]/dt $ 0, to Eq. 7, one finds that

"E:S#2 ! ET 1Km 1 S! "# $

"E:S#1ET S! "

$ 0; (8a)

or

"E:S# $2ET S

! "

ET 1Km 1 S! "

1%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%ET 1Km 1 S

! "# $2! 4ET S! "q :

(8b)

Equation 8b as the solution of Eq. 8a is a standard formula innumerical analysis to avoid a cancellation error when ET"S# ismuch smaller than %ET1Km1"S#&2: Borghans et al. (13)called Eq. 8 the TQSSA.Tzafriri and Edelman (15) derived the following sufficient

condition for the validity of TQSSA:

e%ET; S0& ' 1; (9)

where S0 $ S01"E:S#0; e%ET; S0&$%k2=2k1S0&f %g%ET; S0&&;f %g&$%1!g&!1=2 ! 1; and g%ET; S0& $ 4ETS0%Km1ET1S0&!2: Tzafriri and Edelman also showed that

e%ET; S0& # e%Km; 0& $ 0:25 11k!1

k2

& '!1

: (10)

Therefore, the condition for validity of TQSSA, Eq. 9, issatisfied if k!1 ( k2: From Eq. 10, it is clear that the con-dition is also satisfied to some extent even when k!1 $ 0:So TQSSA seems to be a good approximation for any ratioof ET to S0 and for any values of the rate constants.To examine the stochastic characteristics of a Michaelis-

Menten reaction mechanism when enzyme and substrate arein comparable concentrations, we turn to the chemical masterequation, which describes the evolution of the probabilitydistribution of each potential state of the system. Undernormal circumstances, this probability distribution functionwould be written in terms of the numbers of molecules ofS and E:S (i.e., NS and NE:S), but to isolate the slow reaction,according to the idea of TQSSA, we use the random variablesNS and NE:S: The equation for the probability of the stateNS $ s; NE:S $ es is given by

d

dtP%s; es; t& $ !"k1%s! es&%eT ! es&1 %k!1 1 k2&es#P%s; es; t&

1 k1%s! es1 1&%eT ! es1 1&P%s; es! 1; t&1 k!1%es1 1&P%s; es1 1; t&1 k2%es1 1&P%s1 1; es1 1; t&: (11)

3564 Barik et al.

Biophysical Journal 95(8) 3563–3574

Allowing for a slight abuse of notation, es refers to thenumerical value of the random variable NE:S; not the productof two numbers, e and s. In addition, we use an upper-caseitalic letter to denote the concentration of a particular speciesand a lower-case italic letter to denote its number of mole-cules in a specified volume (e.g., the volume of a yeast cell).For example, ET $ total concentration of enzyme, whereaseT $ total number of enzyme molecules.To obtain an equation for the probability of the composite

variable, NS; we write the joint probability as P%s; es; t& $P%s; t&P%esjs; t& at each occurrence of this expression in Eq.11 and sum Eq. 11 over all possible values of es: Denotingthe conditional expectation of a function of the complexE:S by

Æ f %es&js; tæ $ +N

es$0

f %es&P%esjs; t&; (12)

the master equation for the slow variable becomes

d

dtP%s; t& $ k2Æesjs1 1; tæP%s1 1; t& ! k2Æesjs; tæP%s; t&;

(13)

because all terms relating to the fast reactions cancel out. Forcases of large timescale separation, the distribution of esshould reach steady state quickly compared to the evolutionof s and we can eliminate the dependence on t in the expectedvalue of es in Eq. 13. In this case, Eq. 13 exactly describes thenoise in the chemical reaction

S/k2Æesjs æ

P: (14)

Since in most cases we expect the differential equationdescription of chemical reactions to track the mean values ofthe reactants, it seems reasonable to replace Æesjsæ by thesteady-state value of es from the TQSSA, given by

es2 ! %eT 1Km 1 s&es1 eTs $ 0: (15)

To investigate the accuracy of our approach, we computethe exact value of Æesjsæ and compare it to the TQSSA value.Assuming that s is constant on the short timescale, the evo-lution of the probability of NE:S is governed by the equation

d

dtP%es; t& $ !"k1%s! es&%eT ! es&1 k!1es#P%es; t&

1 k1%s! es1 1&%eT ! es1 1&P%es! 1; t&1 k!1%es1 1&P%es1 1; t&

$ DJ%es1 1; t& ! DJ%es; t&; (16)

where DJ%es; t& $ k!1 esP%es; t&! k1%s! es11&%eT! es11&P%es! 1; t&:At steady state (on the fast timescale),DJ%es& is aconstant, and (considering the case es$ 0) the constant mustbe zero. This equation provides a recurrence relation forP%es& whose solution is

P%es& $Yes

i$1

k1k!1

%s! i1 1&%eT ! i1 1&i

( )P%0&

0# es#min%s; eT&: (17)

The condition that the total probability sums to 1 allows thedetermination ofP%0&; and then themoments Æesjsæ and Æes2jsæcan be computed exactly. Fig. 1 a compares the function Æesjsæwith the deterministic TQSSA result for the case eT $ 10withk1 $ k!1 $ 1:0 and k2 $ 0:1 (arbitrary units). Clearly thestochastic mean Æesjsæ is very close to the deterministic valueof es given by Eq. 15, even for this case of very small numbersof molecules. The variance Æ%es! Æesæ&2jsæ is shown in Fig.1 b and is quite small, on the order of one molecule or less.This analysis is quite similar to that in Goutsias (10) and

Cao et al. (11,12). Our approach differs in using Eq. 15 for themean value of the complex. Goutsias and Cao assume that theassociation and dissociation of the enzyme-substrate complexreaches equilibrium independently of the substrate-to-productconversion reaction, and so their value for Æesjsæ will not de-pend on the slow rate constant k2;whereas our value does. Forcases of extreme timescale separation (k!1 . 100 k2; say), allthese approaches tend to the same solution, and Æesjs; tæ in Eq.15 will tend quickly to a constant independent of k2: For lessextreme cases, the term Æesjs; tæ approaches a steady valuedependent on both fast and slow reactions. Only the TQSSAcaptures this average value accurately, and the reaction (Eq.

FIGURE 1 Michaelis-Menten kinetics. (a) Comparison of steady-state

mean of enzyme-substrate complex as a function of s for eT $ 10: The solidline plots Æesjsæ calculated from the steady-state probability distribution,P%es&; and the dashed line plots deterministic TQSSA results from Eq. 12.

(b) Steady-state variance of the enzyme-substrate complex as a function of s;Æ%es! Æesæ&2jsæ; calculated from P%es&:

Stochastic Simulation of Reactions 3565

Biophysical Journal 95(8) 3563–3574

14), using the corresponding time-invariant propensity, willsatisfy a master equation that approximates the actual masterequation (Eq. 11). Thus the TQSSA is expected to be moreaccurate than the other approaches in cases where the time-scale separation is not large. An example of this behavior isshown in the Supplementary Material (Data S1).We examine the accuracy and efficiency of stochastic

TQSSAby simulating both the full reactionmechanism (Eq. 1)(we call this ‘‘full Gillespie simulation’’) and the simplifiedreaction (Eq. 14), with propensity k2Æesjsæ given by thesteady-state value of es from the TQSSA expression (Eq. 15).We do the comparison for different initial numbers of enzymeand substrate molecules. Of two roots obtained from thequadratic equation (Eq. 15), we take only the root thatsatisfies 0# es# eT: We have chosen k1 $ k!1 $ 1:0 andk2 $ 0:1(arbitrary units), and the numerical simulations areaveraged over 50,000 realizations. In Fig. 2, we plot the meannumber of product Æpæ and its standard deviation as a functionof time for two different cases: 1), eT ( s0 and 2), eT $ s0:From the figure, it is clear that in both cases the stochasticTQSSA is in excellent agreement with the full Gillespiesimulation. The stochastic QSSA proposed by Rao and Arkindoes not give correct results in the two cases considered here.(For the case eT ' s0 (not shown), stochastic QSSA is ac-curate, as expected.) The approximation proposed by Gout-sias gives good results in the cases we consider, but our resultsare more accurate (see the Supplementary Material, Data S1,Fig. S2, a–c). As expected, the stochastic TQSSA simulationis verymuch faster than the full Gillespie simulation (15 timesfaster in the cases considered here).

GOLDBETER-KOSHLAND SWITCH

We now extend our stochastic TQSSA method to coupledenzymatic reactions: the well-known Goldbeter-Koshland(GK) ultrasensitive switch (16). The GK switch consists of asubstrate-product pair (S and Sp) that are interconverted bytwo enzymes (D and E). One can imagine that Sp is thephosphorylated form of the protein S, enzyme E is a kinaseacting on S, and enzyme D is a phosphatase acting on Sp. Thesix reactions involved are

D1 Sp!kd1

kd!1

D :Sp/kd2

S1D (18)

E1 S!ke1

ke!1

E :S/ke2

Sp 1E: (19)

If these reactions (Eqs. 18 and 19) can be described byMichaelis-Menten rate laws, then the time evolution of theswitch is given by

dx

dt$ Ve%1! x&

Je 1 1! x! Vdx

Jd 1 x; (20)

where x $ Sp%t&=ST; ST $ S01"Sp#0; Je $ Kme=ST; Jd $Kmd=ST; Ve $ ke2ET=ST; and Vd $ kd2DT=ST; and Kme andKmd are standard Michaelis constants for enzymes E and D.The superscript on a rate constant indicates the enzyme andthe subscript indicates the particular reaction. The conditionsfor Michaelis-Menten rate laws to apply are ET ' S01Kme

and DT ' "Sp#01Kmd: The steady-state solution of Eq. 20 isknown as the Goldbeter-Koshland function:

The steady-state response of Sp as a function of ET shows asteep sigmoidal shape if Je ' 1 and Jd ' 1: The steepnessoriginates from the kinetic interplay of the two substrate-saturated enzymes acting in opposite directions.The classic GK ultrasensitive switch requires that substrate

concentrations be much larger than enzyme concentrations,which can be expected for metabolic control networks, whereconcentrations of metabolites are much larger than those of

FIGURE 2 Michaelis-Menten kinetics. Time evolution of mean product

Æpæ (solid line) 6 1 SD (dashed lines) calculated from the full Gillespie

simulation. (a) eT $ 1000 and s0 $ 100: (b) eT $ 10 and s0 $ 10: The linesfor stochastic TQSSA simulations are indistinguishable from the fullGillespie simulations.

Sp

ST

$ G%Ve;Vd; Je; Jd& $2VeJd

Vd ! Ve 1VdJe 1VeJd 1%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Vd ! Ve 1VdJe 1VeJd&2 ! 4%Vd ! Ve&VeJd

q : (21)

3566 Barik et al.

Biophysical Journal 95(8) 3563–3574

enzymes. But the applicability of Michaelis-Menten kineticsis limited in the context of protein interaction networks, be-cause a particular enzyme may act on multiple substrates, aparticular substrate may be acted upon by multiple enzymes,and enzymes and substrates often alternate their roles. Whenthe enzyme and substrate swap roles, Michaelis-Menten ki-netics is no longer applicable, since the condition ET 'S01Km cannot possibly be true for both reactions simulta-neously. In this case it is better to write the rate equations interms of "Sp# $ "Sp#1"D :Sp#

d Sp

! "

dt$ ke2"E :S# ! kd2"D :Sp#; (22)

d"D :Sp#dt

$ kd1 DT ! "D :Sp## $

Sp

! "! "D :Sp#

# $

! kd!1"D :Sp# ! kd2 "D :Sp#; (23)

d"E :S#dt

$ ke1%ET ! "E :S#& ST ! Sp

! "! "E :S#

# $

! ke!1"E :S# ! ke2"E :S#; (24)

where ST $ "Sp#1"S# and "S# $ "S#1"E :S#: Invoking theTQSSA on Eqs. 23 and 24, as done by Ciliberto et al. (14),we find that the reactions portrayed in Eqs. 18 and 19 aregoverned by a single ‘‘slow’’ differential equation (Eq. 22)with "E :S# and "D :Sp# given as functions of "Sp# by thesolution of the pair of quadratic equations:

"D :Sp#2 ! DT 1Kmd 1 Sp

! "# $"D :Sp#1DT Sp

! "$ 0; (25)

"E:S#2! ET1Kme1ST! Sp

! "# $"E:S#1ET ST! Sp

! "# $$ 0;

(26)

with 0 # [D:Sp] # DT and 0 # [E:S] # ET.As before, we can write a chemical master equation for the

entire system and then sum over the fast variables to get amaster equation for the slow composite variable NSp

$ sp:

d

dtP%sp; t& $ ! ke2Æesjsp; tæ1 kd2Ædspjsp; tæ

! "P%sp; t&

1 kd2Ædspjsp 1 1; tæP%sp 1 1; t&1 ke2Æesjsp ! 1; tæP%sp ! 1; t& (27)

(for details, consult the Supplementary Material, Data S1).Assuming that the expected values equilibrate on a fastertimescale, we can reduce Eq. 27 to a master equation for thereactions

Sp!kd2 Ædsp jspæ

ke2ÆesjspæS; (28)

with propensities kd2Ædspjspæ and ke2Æesjspæ for the forward andbackward reactions, respectively. If we look at the evolutionof the fast variables in between transitions of the slowvariables, the equations for es and dsp are analogous to theequation for es in the simple enzyme-catalyzed reaction.

For the stochastic TQSSA, numerical simulation of thereactions in Eq. 28 is carried out using Gillespie’s stochasticsimulation algorithm. At each time step, we calculate thepropensities by solving the quadratic equations (Eqs. 25 and26) in terms of numbers of molecules rather than concen-trations. Of the multiple roots obtained from the solution ofthe quadratic equations we take only that root for which0# es# eT and 0# dsp # dT: To compare the full Gillespiesimulation involving six reactions (Eqs. 18 and 19) with thestochastic TQSSA simulation, we consider the parameter setsT $ 900 molecules, kd1 $ 0:05555; kd!1 $ 0:83; kd2 $ 0:17;ke1 $ 0:05; ke!1 $ 0:8; and ke2 $ 0:1 (all the rate constantswith subscript ‘‘1’’ have the unit molecule!1 min!1, and theother rate constants have the unit min!1). These parametervalues are consistent with Ciliberto et al. (14). In Fig. 3, weplot the time evolution of mean Æspæ for three different totalnumbers of kinase and phosphatase: (a), eT $ 45; dT $ 9;(b), eT $ 450; dT $ 90; and (c), eT $ 4500; dT $ 900: Ineach case, the middle line corresponds to the mean value ofSp and the lines above and below are 6 s (1 SD) from themean. For the initial condition, we used NSp

%0& $ 900;NS%0& $ 0; ND:Sp%0& $ 0; and NE:S%0& $ 0: To get the mean,we have averaged each case over 50,000 trajectories. In Fig.3, we have plotted the results from both the full Gillespie andstochastic TQSSA simulations; the agreement is so good thatit is difficult to distinguish between the two results.Fig. 4 compares the steady-state probability distributions

of NSpcalculated from full Gillespie and stochastic TQSSA

simulations. We have chosen dT $ 90 and eT $ 450; withthe same initial conditions as used previously. The agreementbetween the two distributions is so good that it is difficult todistinguish between the two lines. We have found that thesteady-state probability calculated from stochastic TQSSAmatches very well with full Gillespie simulations for both Spand S for any numbers of total enzymes, dT and eT:

FIGURE 3 Goldbeter-Koshland switch. Time evolution of mean Æspæ(solid line) 6 1 SD (dashed lines) for three different total numbers of

enzyme molecules: (a) eT $ 45; dT $ 9; (b) eT $ 450; dT $ 90; and (c)eT $ 4500; dT $ 900: Results of the full Gillespie and stochastic TQSSA

simulations are indistinguishable.

Stochastic Simulation of Reactions 3567

Biophysical Journal 95(8) 3563–3574

In Fig. 5, we compare full Gillespie simulations with sto-chastic TQSSA for steady-state means and standard devia-tions of Sp and D:Sp. We choose the same parameters asbefore, with sT $ 900; dT $ 90 and with different values ofeT: It is clear that for any values of initial enzyme and sub-

strate concentrations, stochastic TQSSA always predicts themean accurately for both Sp and the complex D:Sp. It alwayspredicts the standard deviation for Sp accurately. For thecomplex, the standard deviation of D:Sp predicted by sto-chastic TQSSA is generally good. It fails for the cases wherethe number of free enzyme molecules is very small (see Fig.5 b, ET $ 180 and 270; see also Table S3 in the Supple-mentary Material, Data S1). Fig. 5 a also illustrates theswitching behavior of Sp with increasing eT: (In the Sup-plementaryMaterial (Data S1), we present more comparisonsof full Gillespie with stochastic TQSSA simulations.)In Fig. 6, we report the time autocorrelation function, C%t&;

of steady-state fluctuations around the mean for differentvariables. We define the autocorrelation function for anyvariable x as

Cx%t& $Æ%x%t& ! xss&%x%t1 t& ! xss&æ

Æ%x%t& ! xss&2æ; (29)

where xss is the steady-state value of x: To get a smoothprofile for an autocorrelation function, we averaged resultsover 10,000 trajectories. In general, the autocorrelationfunctions for various species in this network decay exponen-tially. In Fig. 6 awe show that the autocorrelation function ofSp calculated from stochastic TQSSA simulations matchesvery well with full Gillespie simulations. In Fig. 6 b, wecompare autocorrelation functions for free species Sp.TQSSA always predicts a single exponential decay of theautocorrelation function, but in some cases, full Gillespiecalculations show biexponential decay with a short and along component of the decay. From the figures, it is clear thatstochastic TQSSA predicts the long time component accu-rately but it can’t predict the short time component. We find asimilar trend for the complex D:Sp (Fig. 6 c) as well. Thefailure of stochastic TQSSA simulations to capture the fasttimescale of decaying autocorrelations is to be expected,because all fast-timescale events are averaged out in thequasi-steady-state approximations (Eqs. 25 and 26).

GENERAL FORMULATION OF THE TQSSA

To provide a general setting for the TQSS approach to sto-chastic simulation taken in this article, we first develop ageneral framework for the deterministic TQSSA and thenextend it to a stochastic formulation. Systems of enzyme-catalyzed reactions, the focus of this article, fit naturally intothis general framework, but the framework encompasses amuch broader class of systems.Consider a chemical reaction network with N species par-

ticipating in M independent reactions. The deterministicchemical rate equations can be written as dx/dt $ S)r(x(t)),where x(t) is an N-vector of concentrations of reactingchemical species, S is an N 3 M matrix of stoichiometriccoefficients, and r is a vector of time-dependent fluxesthrough the M reactions of the network. (These fluxes are, in

FIGURE 4 Goldbeter-Koshland switch. Steady-state probability distribu-tions of Sp for sT $ 900; dT $ 90; and eT $ 450: The solid line represents

the full Gillespie, and the dashed line the stochastic TQSSA.

FIGURE 5 Goldbeter-Koshland switch. Steady-state mean (bars) and

standard deviation (error bars) of Sp (a) and D :Sp (b) for sT $ 900 and

dT $ 90: The open bars represent the full Gillespie, and the hatched bars thestochastic TQSSA. In some cases, the standard deviations in stochasticTQSSA are so small that the error bars are not visible.

3568 Barik et al.

Biophysical Journal 95(8) 3563–3574

general, nonlinear functions of the concentrations of the re-acting species.) We assume that the reactions have beenpartitioned into Ms slow reactions and Mf fast reactions, r $* rsrf

+;whereMs1Mf$M. If the reactions are ordered in this

fashion, then the stoichiometric matrix can be separated intotwo submatrices, S $ [SslowSfast], where Sslow is an N 3 Ms

matrix of stoichiometric coefficients for the slow reactions,and Sfast is an N 3 Mf matrix of stoichiometric coefficientsfor the fast reactions.Although it may be possible to separate reactions into slow

and fast categories, it is usually impossible to separatechemical species into these categories because most chemicalspecies participate in both fast and slow reactions. Using theTQSS idea, we seek linear combinations of the originalchemical species (e.g., xs $ lT ) x; where l is an N 3 Ns

matrix of constants) such that dxs=dt involves only slowreactions, rs. In such a case,

dxs=dt $ lT dx=dt $ lT Sslow Sfast" # rsrf

, -$ "Ss 0#

rsrf

, -: (30)

Hence, the condition for a slow combination of species, xs $lT ) x; is that the columns of l span the left null space of theN3Mf matrix Sfast. The number of slow variables, Ns, is the di-mension of this left null space. A linear transformationmatrix L can be constructed by choosing the first Ns rows ofL as appropriate linearly independent components of thenull space, and the remaining rows as standard basis vectorsthat are linearly independent of the first Ns rows. (It is clearthat L is not unique, and this freedom can be used for other

purposes.) The transformationh xsxf

i$ Lx produces chemical

rate equations of the form (the hat is dropped for notationalsimplicity):

d

dtxs%t&xf%t&

, -$ Ss 0

Ssf Sff

, -) rs%xs; xf&

rf%xs; xf&

, -(31)

where the slow variables, xs; participate only in slow reac-tions, and the fast variables, xf ; are governed by both fast andslow reactions. The 0 matrix in Eq. 31 has Ns rows andMf columns, and the stoichiometry matrix is equal toL)[SslowSfast].In terms of the preceding examples, the association and

dissociation of enzyme-substrate complexes are chosen asthe fast reactions, xs corresponds to the total variables definedby TQSSA (denoted with hats in our examples), and xf cor-responds to the original complexes in the system. Any con-servation relations in the system show up as all zero rows ofSs and can be eliminated from the differential equation. Byconstruction there are no linear combinations of species thatare governed only by slow reactions among the Nf $ N – Ns

‘‘fast’’ variables.In the deterministic formulation of TQSSA, the derivatives

of the fast variables are set equal to zero and the resultingequations solved for the quasi-steady-state values of the fastvariables in terms of the slow variables. This in turn allowsfor the simulation of the slow subsystem with the values forthe fast variables replaced by their quasi-steady-state values.To see how this idea transfers to the stochastic setting,

consider the chemical master equation for the entire system(dropping the bold face):

FIGURE 6 Goldbeter-Koshland switch. Autocorrelation functions of

fluctuations for the species (a) Sp; (b) Sp; and (c) D :Sp; calculated from

full Gillespie simulation (solid lines) and from stochastic TQSSA simulation(dashed lines) for different numbers of enzymes: eT $ 9; dT $ 9 (black);eT $ 45; dT $ 9 (red); and eT $ 90; dT $ 90 (blue).

Stochastic Simulation of Reactions 3569

Biophysical Journal 95(8) 3563–3574

@P%xs;xf ; t&@t

$ +islow

ai%xs!Ss;i;xf !Ssf;i&P%xs!Ss;i;xf ! Ssf;i; t&

1 +ifast

ai%xs;xf !Sff;i&P%xs;xf !Sff;i; t&

! +islow

ai%xs;xf&P%xs;xf ; t&!+ifast

ai%xs;xf&P%xs;xf ; t&;

(32)

where ai is the propensity of the i th reaction. WritingP%xs; xf ; t& $ P%xf jxs; t&3P%xs; t& in the above and summingover the fast species gives

Defining

Æai%xs; xf&jxs; tæ $ +xf

ai%xs; xf&P%xf jxs; t&; (34)

and recognizing that Æai%xs; xf&jxs; tæ $ Æai%xs; xf ! a&jxs; tæ;allows us to write

@P%xs; t&@t

$ +islow

Æai%xs ! Ss;i; xf ; t&jxs ! Ss;iæP%xs ! Ss;i; t&

! +islow

Æai%xs; xf&jxs; tæP%xs; t&: (35)

This can be recognized as the master equation of the sys-tem consisting of just the slow reactions involving thespecies xs; having stoichiometric matrix Ss and propensitiesÆai%xs; xf&jxs; tæ:If the original slow reaction rates depend at most linearly

on the fast variables xf ; which will be the case if no slowvariable is the result of a bimolecular reaction where bothreactants are fast variables, then Æai%xs; xf&jxs; tæ will dependonly on the slow variables and the means of the fast variablesgiven the values of the slow variables. (This was the majorrestriction in Goutsias (10).) We make the further assumptionthat the differential equations governing the system provideaccurate information concerning the means of the variables.Then, assuming that the fast variables go to steady statequickly compared to the evolution of the slow variables (thestandard TQSSA assumption), we can set the derivatives ofthe fast variables to zero, yielding

Ssfrs%xs; xf&1 Sffrf%xs; xf& $ 0: (36)

Equation 36 provides Nf coupled equations in N un-knowns. Given the assumption of mass-action kinetics, these

equations will be at most second order in the variables, xs andxf. In most cases, we can solve these equations for the Nf fastvariables in terms of the slow variables, providing us with themeans of the fast variables needed to compute the rates of theslow reactions of the reduced stochastic system. Clearly, aspointed out by Goutsias (17), the solution of the deterministicchemical reaction equations can differ from the mean valuesof the solution of the stochastic reaction equations when themolecular populations approach zero. Although not a con-cern for the problems considered in this article, this potentiallimitation must be kept in mind for the general case.

COUPLED GK SWITCHES (BISTABILITY)

We now apply this general approach to coupled GK switches(Fig. 7 a). To reactions shown in Eqs. 18 and 19, we add asecond GK switch, which converts unphosphorylated E to aphosphorylated form Ep and back. The conversion of E to Ep

is catalyzed by S, and the dephosphorylation reaction of E iscatalyzed by the phosphatase, F. In this network, S and E actas mutual antagonists. This network describes the transitionfrom the G2 phase to mitosis (the M phase) in the eukaryoticcell cycle, where S $ MPF (M-phase promoting factor, adimer of Cdc2 and cyclin B) and E $ Wee1 (a kinase that

@P%xs; t&@t

$ +xf

+islow

ai%xs ! Ss;i; xf ! Ssf;i&P%xf ! Ssf;ijxs ! Ss;i; t&P%xs ! Ss;i; t&

1 +ifast

ai%xs; xf ! Sff;i&P%xf ! Sff;ijxs; t&P%xs; t&

! +islow

ai%xs; xf&P%xf jxs; t&P%xs; t&

!+ifast

ai%xs; xf&P%xf jxs; t&P%xs; t&

8>>>>>>><

>>>>>>>:

9>>>>>>>=

>>>>>>>;

: (33)

FIGURE 7 Bistability for coupled GK switches. (a) Reaction scheme. (b)Bifurcation diagram.

3570 Barik et al.

Biophysical Journal 95(8) 3563–3574

phosphorylates and inactivates Cdc2). Novak and Tyson (18)used this module in a mathematical model for the G2-to-Mtransition during early embryonic cell cycles of Xenopus.Using GK rate laws, like Eq. 20, they showed that the con-centration of S may exhibit two stable steady-state values,"S# * 0 and "S# * ST; for intermediate values of ST: That is tosay, the control system (Fig. 7 a) can exhibit bistability for STin a certain interval (S1 , ST , S2). Ciliberto et al. (14)showed that bistability is maintained after unpacking thereaction network into elementary steps, provided that Sp hassome ability to bind E but limited ability to convert E to Ep.To apply the TQSSA to the resulting network, they intro-duced the definitions

S $ S1 "E :S#1 "S :E#E $ E1 "E :S#1 "S :E#1 "Sp :E#ST $ S1 Sp 1 "E :S#1 "S :E#1 "D :Sp#1 "Sp :E#DT $ D1 "D :Sp#FT $ F1 "F :Ep#: (37)

In terms of these variables, the TQSSA equations are

dS

dt$ kd2 "D :Sp# ! ke2"E :S#

dE

dt$ kf2"F :Ep# ! kb2 "S :E# ! kb92 "Sp :E# (38)

and

Using these equations and the parameter values given inTable 1 (taken from Marlovits et al. (19)), Ciliberto et al.found that s is a bistable function of sT (Fig. 7 b).

In the same way as before, we can obtain a master equationfor the slow reactions:

Sp!kd2 Ædspæ

ke2ÆesæS

E !kb2 Æseæ1kb2

9Æspeæ

kf2ÆfepæEp; (40)

with propensities based on the expected values of the num-bers of enzyme-substrate complexes. We calculate the ex-pected values of the complexes by solving the coupledalgebraic equations (Eq. 39). In Fig. 8, we compare thesteady-state probability distribution of S obtained from fullGillespie simulations of this reaction network with stochasticTQSSA simulations of Eq. 40. We calculate the steady-stateprobability distributions for different values of sT: In themonostable zone (sT , 88), the prediction of stochasticTQSSA is good (Fig. 8 a). As sT increases into the bistablezone, the prediction of stochastic TQSSA is still fine (Fig.8 b), although the relative intensities of the two peaks in thebistable zone differ from the ‘‘exact’’ result of full Gillespiesimulations. In the upper monostable region (sT . 99), theprobability distribution calculated from the stochasticTQSSA is a little shifted from the exact result (Fig. 8 c). Inthe deterministic case, the system settles to a particular steadystate depending on initial conditions. In the stochastic case,the system jumps back and forth between the alternativestable steady states.

In Fig. 9, we plot a sample trajectory of S in the bistableregime (sT $ 95); we initialize the system with s $ 8; sp $87 molecules. The upper panel shows a trajectory calculatedfrom a full Gillespie simulation and the lower panel shows astochastic TQSSA simulation. It appears from this examplethat the residence time in the upper steady state is muchshorter in the stochastic TQSSA calculation than in the fullGillespie calculation. To investigate the apparent discrep-ancy, we calculate the residence time distributions of lowerand upper steady states in both full Gillespie and stochasticTQSSA simulations. We initialize the system at the lowersteady state (s $ 8) and record the time t0 when s%t& reachesfor the first time its upper steady-state value (s%t0& $ 58).Then we follow the trajectory until s%t& next reaches its lowersteady-state value (s%t1& $ 8). Continuing this process, werecord a sequence of times t0; t1; t2 . . . : when s%t& flips back

"D :Sp#2 ! %DT 1Kmd 1 ST ! S! "Sp :E#&"D :Sp#1DT%ST ! S! "Sp :E#& $ 0

"E :S#2 ! %E! "S :E# ! "Sp :E#1Kme 1 S! "S :E#&"E :S#1 %E! "S :E# ! "Sp :E#&%S! "S :E#& $ 0

"S :E#2 ! %E! "E :S# ! "Sp :E#1Kmb 1 S! "E :S#&"S :E#1 %E! "E :S# ! "Sp :E#&%S! "E :S#& $ 0

"Sp :E#2 ! %ST ! S! "D :Sp#1Kmb9 1 E! "E :S# ! "S :E#&"Sp :E#1 %ST ! S! "D :Sp#&%E! "E :S# ! "S :E#& $ 0

"F :Ep#2 ! %FT 1Kmf 1ET ! E&"F :Ep#1FT%ET ! E& $ 0 (39)

TABLE 1 Parameter values for coupled GK switches

Enzyme S Dp Sp E F

Substrate E Sp E S Ep

Letter, l b d b9 e f

kl1 0.625 0.0001125 0.00625 0.0125 0.00125

kl!1 10.6 0.005 0.005 0.05 0.001

kl2 0.4 0.085 0.0001 0.05 0.2Kml 17.6 800 0.816 8 160.8

LT 1600 160 40

The association rate constants (kl1) are in units molecule!1 min!1, and

dissociation (kl!1) and catalytic (kl2) rate constants are min!1. Units forMichaelis-Menten constants (Km) are molecules. LT is the total number of

molecules of enzymes D, E, and F.

Stochastic Simulation of Reactions 3571

Biophysical Journal 95(8) 3563–3574

and forth between its upper and lower steady states. We de-fine (t1 ! t0) as a sample residence time in the upper steadystate, (t2 ! t1) as a sample residence time in the lower steadystate, and so on. From a long simulation, we compute manysample residence times for both steady states. In Fig. 10, weplot cumulative distributions of these residence times. Thetop panel compares the cumulative probability distributionfor residence times in the upper steady state from fullGillespie and stochasticTQSSA simulations (Fig. 10, a and b,respectively). The dashed lines are single exponential fits tothe cumulative distribution. The bottom panel is the samefor the lower steady state. Though the stochastic TQSSAcalculation produces an exponential decay distribution, the

mean residence times for the upper (TU $ 42.15 3 103 min)and lower (TL $ 5.55 3 103 min) steady states do not agreewith the full Gillespie simulation (TU $ 129.96 3 103 min;TL $ 20.84 3 103 min).Suspecting that these divergent results might be attribut-

able to poor timescale separation (see the SupplementaryMaterial, Data S1) in our parameter set (Table 1), we repeatedthe simulations for a new parameter set with large rate con-stants for association and dissociation of enzyme-substratecomplexes, while keeping the rate constant for enzymaticconversion of substrate to product unchanged (see the Sup-plementary Material, Table S6, Data S1). In this case, wefind that stochastic TQSSA simulations are in excellentagreement with full Gillespie simulations with respect tosteady-state probability distributions (see the SupplementaryMaterial, Fig. S4, a–c, Data S1) and mean residence times inthe lower and upper steady states.

DISCUSSION

The stochastic TQSSA approach differs from tau-leaping andchemical Langevin approaches in that it does not simulate thefast reactions. In tau-leaping (3–6), the number of fast-reac-tion events that occur during each time interval is estimatedusing a Poisson random variable, whereas Haseltine andRawlings (8) simulate the fast reactions using chemicalLangevin equations. These methods can work for enzyme-catalyzed reactions, but tau-leaping can be slow for stiffsystems and the accuracy of the chemical Langevin equationcan be a problem when the number of molecules of reactingspecies becomes small.

FIGURE 8 Bistable switch. Steady-state probability distributions of S:The solid lines represent the full Gillespie, and the dashed line the stochastic

TQSSA. (a) sT $ 86; (b) sT $ 95; and (c) sT $ 100:

FIGURE 9 Bistable switch. The trajectory of S in the bistable zone

(sT $ 95). (Upper) Full Gillespie simulation. (Lower) Stochastic TQSSAsimulation.

3572 Barik et al.

Biophysical Journal 95(8) 3563–3574

Other approaches rely on the separation of timescales thatoften occurs in enzyme-catalyzed reactions. Rao and Arkin(9) separate the species according to whether they evolve onfast or slow timescales and show how to obtain a masterequation for the slow species that depends on the averagevalues of the fast species. Furthermore, assuming that thetimescale separation allows the fast species to equilibratequickly, they show that in the Michaelis-Menten case themaster equation for the slow species corresponds to the re-duced reaction obtained using the quasi-steady-state ap-proximation (QSSA). Thus, simulating the reduced reactionusing Gillespie’s stochastic simulation algorithm will accu-rately reproduce the noise in the reaction system whileavoiding the bottleneck created by the fast reactions.Goutsias (10) takes a similar approach, but separates the

reactions, not species, according to timescale. His approach ismore general than Rao and Arkin in terms of how he deter-mines the average values of the fast species, and for theMichaelis-Menten case his approach is not limited by theconditions under which QSSA is valid. In the general case,

his approach requires the solution of a system of quadraticequations subject to a linear inequality constraint after eachoccurrence of a slow reaction.Cao et al. (11,12) also identify fast- and slow-reaction

channels, but propose a framework using the steady-stateprobability function for the fast species to calculate the mo-ments of the fast species. Recognizing that this calculationwill not always be feasible, they explore several approximatemethods, the simplest of which involves using equilibriumvalues from the associated reaction-rate equations.The TQSSA approach differs from the quasi-equilibrium

assumption of Goutsias (10) and Cao (11,12) in that it allowsthe slow-timescale reactions to impact the steady state of thefast reactions. For the case of large timescale separation, allthree approaches will give comparable results. This case isnice theoretically since the average values of the complexesdecay quickly to their steady-state values after the firing of aslow reaction, and the master equation governing the slowreactions using the steady-state average propensities is agood approximation to the actual master equation that hastime-varying coefficients. In cases with less timescale sepa-ration, the time-varying coefficients of the actual masterequation do not settle out between firings of the slow reac-tions. In these cases, the TQSSA approach can still provide agood approximation, as shown in our examples.For the Goldbeter-Koshland ultrasensitive switch, our

stochastic results (means and standard deviations) for theslow variables using TQSSA are all but identical to the resultsobtained by simulating the full system using Gillespie’sstochastic simulation algorithm. In addition, the means forthe complexes, which are not directly simulated, are veryaccurate in all cases, and the standard deviations are accurateexcept in cases where the free enzyme populations are re-duced to only a few molecules. In these problem cases, theinaccuracies in the standard deviations are of the order of1 molecule. This accuracy for variables that are not directlysimulated is surprising. It is due largely to the fact that thestandard deviation is composed of components from both thefast and slow reactions, and the fast (unsimulated) componentis very small (see Fig. 1 b).Furthermore, for slow variables, the TQSSA technique

accurately captures the autocorrelation function of fluctua-tions about a steady state. For the complexes, free enzyme,and free substrate variables that are not directly simulated, theautocorrelation functions can consist of multiple exponentialcomponents, and our approach will find only the slowestcomponents of the autocorrelation functions. This is to beexpected, since all information about fast transients is lost inthe TQSSA approach.Our final example of a coupled Goldbeter-Koshland

switch exhibiting bistability is a particularly sensitive test ofthe accuracy of the stochastic TQSSA, since small changes instochastic trajectories near the unstable steady state canchange the stable steady state that is approached. For theparameter values considered here (Table 1), there is little

FIGURE 10 Bistable switch. Plot of cumulative probability (pt) of resi-dence times (tR) for sT $ 95: (Upper) Upper steady state. (Lower) Lowersteady state. The labeled lines represent full Gillespie (a), and stochastic

TQSSA (b). Dashed lines are best-fitting exponential curves to the cumu-

lative probability distributions.

Stochastic Simulation of Reactions 3573

Biophysical Journal 95(8) 3563–3574

timescale separation between complex formation and productformation. Even so, TQSSA produces reasonably accurateprobability distributions, as seen in Fig. 8, a–c, although theresidence time distribution is inaccurate (Fig. 10).In summary, our results show that stochastic simulations

based on the TQSSA can be implemented in a straightfor-ward, algorithmic manner to efficiently calculate accuratetrajectories for the slow variables in a multiple-timescalesystem of chemical reactions.

SUPPLEMENTARY MATERIAL

To view all of the supplemental files associated with thisarticle, visit www.biophysj.org.

This work has been supported by a grant from the National Institutes ofHealth (GM078989). We also acknowledge generous support from Virginia

Tech’s Advanced Research Computing facility for a grant of computing

resources on System X.

REFERENCES

1. Gillespie, D. T. 1976. A general method for numerically simulating thestochastic time evolution of coupled chemical reactions. J. Comput.Phys. 22:403–434.

2. Gillespie, D. T. 1977. Exact stochastic simulation of coupled chemicalreactions. J. Phys. Chem. 81:2340–2361.

3. Gillespie, D. T. 2001. Approximate accelerated stochastic simulation ofchemically reacting systems. J. Chem. Phys. 115:1716–1733.

4. Cao, Y., L. R. Petzold, M. Rathinam, and D. T. Gillespie. 2004. Thenumerical stability of leaping methods for stochastic simulation ofchemically reacting systems. J. Chem. Phys. 121:12169–12178.

5. Rathinam, M., L. R. Petzold, Y. Cao, and D. T. Gillespie. 2003.Stiffness in stochastic chemically reacting systems: The implicit tau-leaping method. J. Chem. Phys. 119:12784–12794.

6. Cao, Y., D. T. Gillespie, and L. R. Petzold. 2007. The adaptiveexplicit-implicit tau-leaping method with automatic tau selection.J. Chem. Phys. 126:224101–224108.

7. Gillespie, D. T. 1992. A rigorous derivation of chemical masterequation. Physica A. 188:404–425.

8. Haseltine, E. L., and J. B. Rawlings. 2002. Approximate simulation ofcoupled fast and slow reactions for stochastic chemical kinetics.J. Chem. Phys. 117:6959–6969.

9. Rao, C. V., and A. P. Arkin. 2003. Stochastic chemical kinetics and thequasi-steady-state assumption: application to the Gillespie algorithm.J. Chem. Phys. 118:4999–5010.

10. Goutsias, J. 2005. Quasiequilibrium approximation of fast reactionkinetics in stochastic biochemical systems. J. Chem. Phys. 122:184102–184116.

11. Cao, Y., D. T. Gillespie, and L. R. Petzold. 2005. The slow-scalestochastic simulation algorithm. J. Chem. Phys. 122:014116–014123.

12. Cao, Y., D. T. Gillespie, and L. R. Petzold. 2005. Acceleratedstochastic simulation of the stiff enzyme-substrate reaction. J. Chem.Phys. 123:144917–144928.

13. Borghans, J. A., R. J. de Boer, and L. A. Segel. 1996. Extending thequasi-steady state approximation by changing variables. Bull. Math.Biol. 58:43–63.

14. Ciliberto, A., F. Capuani, and J. J. Tyson. 2007. Modeling networks ofcoupled enzymatic reactions using the total quasi-steady state approx-imation. PLoS Comput. Biol. 3:e45.

15. Tzafriri, A. R., and E. R. Edelman. 2004. The total quasi-steady-stateapproximation is valid for reversible enzyme kinetics. J. Theor. Biol.226:303–313.

16. Goldbeter, A., and D. E. Koshland. 1981. An amplified sensitivityarising from covalent modification in biological systems. Proc. Natl.Acad. Sci. USA. 78:6840–6844.

17. Goutsias, J. 2007. Classical versus stochastic kinetics modeling ofbiochemical reaction systems. Biophys. J. 92:2350–2365.

18. Novak, B., and J. J. Tyson. 1993. Numerical analysis of a compre-hensive model of M-phase control in Xenopus oocyte extracts andintact embryos. J. Cell Sci. 106:1153–1168.

19. Marlovits, G., C. J. Tyson, B. Novak, and J. J. Tyson. 1998. ModelingM-phase control in Xenopus oocyte extracts: the surveillance mecha-nism for unreplicated DNA. Biophys. Chem. 72:169–184.

3574 Barik et al.

Biophysical Journal 95(8) 3563–3574