Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General

Dr V. Andasari http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html

Lecture 4b: Continuous-Time Birth and DeathProcesses

Continuous-time Markov chains are stochastic processes whose time is continuous, t ∈ [0,∞),but the random variables are discrete. Prominent examples of continuous-time Markov processesare Poisson and death and birth processes. These processes play a fundamental role in the theoryand applications that embrace queueing and inventory models, population growth, engineeringsystems, etc [3]. Here we discuss the birth and death process with its invariants.

1 Preliminaries

1.1 Probability Generating Functions

The probability generating function is a concept used for distributions with non-negative integers.

Consider a discrete random variable X taking nonnegative values. Let f denote the probabilitymass function (p.m.f.) of X, defined as

fk = Prob{X = k} = Pk , k = 0, 1, 2, · · ·

where∞∑

k=0Pk = 1.

The probability generating function (p.g.f.) of X, denoted �X, is defined as

�X(z) = E[zX] =

∞∑k=0

Pkzk , (1)

for some z ∈ R, or, the pmf,

Pk(t) =1k!

∂k�X(z)∂zk

∣∣∣∣∣∣z=0

. (2)

1

http://people.bu.edu/andasari/courses/stochasticmodeling/stochastics.html


Note that

• The series converges absolutely for |z| ≤ 1:

�X(1) =

∞∑k=0

Pk(1)k

=

∞∑k=0

Pk

= 1 .

Thus, �X(z) is well defined for |z| ≤ 1.

• Also, the p.g.f. generates the probabilities associated with the distribution

�X(0) = P0 , �′

X(0) = P1 , �′′

X(0) = 2!P2 ,

which are resulted from, by taking

�X(z) = P0 + P1z + P2z2 + P3z3 + · · ·

whence at z = 0, then P0 = �X(0). And

�X(z)∂z

= P1 + 2P2z + 3P3z2 + · · ·

whence at z = 0, then P1 = �′

X(0). And so forth.

• In general, the nth derivative of the p.g.f. of X is

�(n)X (0) = n!Pn .

The p.g.f. can be used to calculate the mean and variance of a random variable X:

(1) The first derivative of �X is

�′

X(z) =d�X(z)

dz=

∞∑k=1

kPkzk−1 , for − 1 < z < 1 .

By letting z approach one from the left, z→ 1−, we obtain the mean of X

�′

X(1) =

∞∑k=1

kPk = E[X] = µX .

2



(2) The second derivative of �X is

�′′

X(z) =d2�X(z)

dz2 =

∞∑k=1

k(k − 1)Pkzk−2 ,

so that as z→ 1−,

�′′

X(1) =

∞∑k=1

k(k − 1)Pk = E[X(X − 1)] = E[X2 − X] .

If the mean is finite, then the variance of X is

�′′

X(1) + �′

X(1) −(�′

X(1))2

= E[X2 − X] + E[X] − (E[X])2

= E[X2] − E[X] + E[X] − (E[X])2

= E[X2] − (E[X])2

= Var(X) .

2 Birth and Death Processes

Consider a large class of infinite state space, continuous-time Markov chain whose states are{0, 1, 2, · · · }. Transitions of state may go only from n to n + 1 or from n to n − 1. We can viewthe state of the system as the size of a population that can increase by a “birth” (n → n + 1) ordecrease by a “death” (n → n − 1). Suppose that whenever there are n people in the system,then: (i) new individuals enter the system at an exponential rate λn, and (ii) individuals leave thesystem at an exponential rate µn. That is, whenever there are n people in the system, the time untilthe next arrival is exponentially distributed with mean 1/λn and is independent of the time untilthe next departure which is also exponentially distributed but with mean 1/µn. The parameters{λn}

∞n=0 and {µn}

∞n=1 are called the arrival (or birth) and departure (or death) rates, respectively. If

the population is 0, there are no deaths and µ0 = 0 [5, 7].

2.1 General Birth and Death Process

The continuous-time birth and death Markov chain {X(t) : t ∈ [0,∞)} may have either a finite{0, 1, 2, · · · ,N} or infinite {0, 1, 2, · · · } state space. Assume that its transition probabilities Pi j(t)are stationary, i.e.,

Pi j(t) = Prob{X(t + s) = j | X(s) = i} , for all ∆t ≥ 0 .

3



In addition, assume the infinitesimal transition probabilities for this process are [1]

Pi,i+ j(∆t) = Prob{X(t + ∆t) − X(t) = j | X(t) = i}

=

λi∆t + o(∆t) , j = 1

µi∆t + o(∆t) , j = −1

1 − (λi + µi)∆t + o(∆t) , j = 0

o(∆t) , j , −1, 0, 1 .

(3)

for ∆t sufficiently small, µ0 = 0, λ0 > 0, and λi > 0, µi > 0 for i = 1, 2, · · · . It is often the casethat λ0 = 0, except when there is immigration. The initial conditions are

Pi j(0) = δi j =

1 , i = j

0 , i , j .

In a small time interval ∆t, at most one change in state can occur, either a birth, i → i + 1 or adeath, i→ i − 1 [1].

In the same way as for the Poisson process, we define Pi(t) = Prob{X(t) = i} and assume X(0) =

0. If ∆t > 0, i ≥ 1, by invoking the law of total probability and the Markov property, we obtainfrom the Chapman-Kolmogorov equations that,

Pi j(t + ∆t) =

∞∑k=0

Pik(t) Pk j(∆t)

=

∞∑k=0

Pik(t) · Prob{X(t + ∆t) = j | X(t) = k}

=

∞∑k=0

Pik(t) · Prob{X(t + ∆t) − X(t) = j − k | X(t) = k}

=

k= j∑k=0

Pik(t) · Prob{X(t + ∆t) − X(t) = j − k | X(t) = k} .

Now, for k = j, for the right hand side we have

= Pi j(t) · Prob{X(t + ∆t) − X(t) = j − j | X(t) = j}= Pi j(t) · Prob{X(t + ∆t) − X(t) = 0 | X(t) = j}= Pi j(t) ·

[1 − (λ j + µ j)∆t + o(∆t)

],

4



for k = j − 1,

= Pi, j−1(t) · Prob{X(t + ∆t) − X(t) = j − ( j − 1) | X(t) = j − 1}= Pi, j−1(t) · Prob{X(t + ∆t) − X(t) = 1 | X(t) = j − 1}= Pi, j−1(t) ·

[λ j−1∆t + o(∆t)

],

for k = j + 1,

= Pi, j+1(t) · Prob{X(t + ∆t) − X(t) = j − ( j + 1) | X(t) = j + 1}= Pi, j+1(t) · Prob{X(t + ∆t) − X(t) = −1 | X(t) = j + 1}= Pi, j+1(t) ·

[µ j+1∆t + o(∆t)

],

whereas for k , j, j − 1, j + 1, that is for k ≤ j − 2,

= Pik(t) · Prob{X(t + ∆t) − X(t) ≥ 2 | X(t) = k}= Pik(t) · o(∆t) ,

and k ≥ j + 2,

= Pik(t) · Prob{X(t + ∆t) − X(t) ≤ −2 | X(t) = k}= Pik(t) · o(∆t) .

Collecting all together,

Pi j(t + ∆t) = Pi j(t) ·[1 − (λ j + µ j)∆t + o(∆t)

]+ Pi, j−1(t) ·

[λ j−1∆t + o(∆t)

]+

Pi, j+1(t) ·[µ j+1∆t + o(∆t)

]+ Pik(t) · o(∆t)

= Pi, j−1(t)λ j−1∆t + Pi, j+1(t)µ j+1∆t + Pi j(t)[1 − (λ j + µ j)∆t

]+ o(∆t) ,

which holds for all i and j in the state space with the exception of j = 0 and j = N (if thepopulation size is finite).

If j = 0, then (and due to µ0 = 0)

Pi0(t + ∆t) = Pi1(t)µ1∆t + Pi0(t)[1 − λ0∆t

]+ o(∆t) .

If j = N, which is the maximum population size, then

PiN(t + ∆t) = Pi,N−1(t)λN−1∆t + PiN(t)[1 − µN)∆t

]+ o(∆t) ,

5



where λN = 0 and PkN = 0 for k > N. Subtracting Pi j(t), Pi0(t), and PiN(t) from the precedingthree equations, respectively, dividing by ∆t, and taking the limit as ∆t → 0, yields the forwardKolmogorov differential equations for the general birth and death process,

dPi j(t)dt

= λ j−1Pi, j−1(t) − (λ j + µ j)Pi j(t) + µ j+1Pi, j+1(t)

dPi0(t)dt

= −λ0Pi0(t) + µ1Pi1(t) , for j = 0

dPiN(t)dt

= λN−1Pi,N−1(t) − µN PiN(t) , for j = N .

(4)

The forward Kolmogorov differential equations can be written in matrix notation,

dP(t)dt

= P(t)Q

dPi0/dt

dPi1/dt

dPi2/dt

dPi3/dt

...

=

[Pi0 Pi1 Pi2 Pi3 · · ·

]

−λ0 λ0 0 0 · · ·

µ1 −(λ1 + µ1) λ1 0 · · ·

0 µ2 −(λ2 + µ2) λ2 · · ·

0 0 µ3 −(λ3 + µ3) · · ·

......

......

,

where the generator matrix Q for the infinite state space

Q =

−λ0 λ0 0 0 · · ·

µ1 −(λ1 + µ1) λ1 0 · · ·

0 µ2 −(λ2 + µ2) λ2 · · ·

0 0 µ3 −(λ3 + µ3) · · ·

......

......

, (5)

6



and for the finite state space

Q =

−λ0 λ0 0 0 · · ·

µ1 −(λ1 + µ1) λ1 0 · · ·

......

... · · ·...

0 0 µN−1 −(λN−1 + µN−1) λN−1

0 0 0 µN −µN

. (6)

Similarly, to obtain the backward Kolmogorov differential equations, we start from the Chapman-Kolmogorov equations,

Pi j(∆t + t) =

∞∑k=0

Pik(∆t) Pk j(t)

= Pi,i−1(∆t)Pi−1, j(t)︸︷︷︸k = i−1

+ Pii(∆t)Pi j(t)︸︷︷︸k = i

+ Pi,i+1(∆t)Pi+1, j(t)︸︷︷︸k = i+1

+∑

k

Pik(∆t)Pk j(t) ,

where the last summation is over all k , i − 1, i + 1, i.

For k = i,Pii(∆t)Pi j(t) = Pi j(t) ·

[1 − (λi + µi)∆t + o(∆t)

],

for k = i − 1,

Pi,i−1(∆t)Pi−1, j(t) = Pi−1, j(t) ·[µi∆t + o(∆t)

],

for k = i + 1,

Pi,i+1(∆t)Pi+1, j(t) = Pi+1, j(t) ·[λi∆t + o(∆t)

],

and for k , j, j − 1, j + 1, we have∑k

Pik(∆t)Pk j(t) = Pk j(t) · o(∆t) .

Collecting all together,

Pi j(t + ∆t) =[1 − (λi + µi)∆t

]Pi j(t) + µi∆tPi−1, j(t) + λi∆tPi+1, j(t) + o(∆t) ,

7



or, rearranging,

Pi j(t + ∆t) − Pi j(t) = µi∆tPi−1, j(t) − (λi + µi)∆tPi j(t) + λi∆tPi+1, j(t) + o(∆t) .

Dividing by ∆t and taking the limit as ∆t → 0, we obtain the backward Kolmogorov differentialequations

dPi j(t)dt

= µiPi−1, j(t) − (λi + µi)Pi j(t) + λiPi+1, j(t) ,

dP0 j(t)dt

= −λ0P0 j(t) + λ0P1 j(t) , for i = 0

dPN j(t)dt

= µN PN−1, j(t) − µN PN j(t) , for i = N .

(7)

where µ0 = 0 and λN = 0, and j ≥ 0, i ≥ 0.

The backward Kolmogorov differential equations can be written in matrix notation,

dP(t)dt

= QP(t)

dP0 j/dt

dP1 j/dt

dP2 j/dt

...

dPN j/dt

=

−λ0 λ0 0 0 · · ·

µ1 −(λ1 + µ1) λ1 0 · · ·

0 µ2 −(λ2 + µ2) λ2 · · ·

......

......

0 0 0 µN −µN

P0 j

P1 j

P2 j

...

PN j

.

2.2 Stationary Probability Distribution (The Limiting Behavior of Birthand Death Processes)

For a general birth and death process that has no absorbing states, the limits

limt→∞

Pi j(t) = π j ≥ 0 (8)

exist and are independent of the initial state i. The limits form the limiting distribution (orlimiting probability) of the process, or also known as a stationary probability distribution. For

8



a more detailed discussion on this topic, see subsection 1.7 of Lecture 4a (Continuous-TimeMarkov Chain Models). To determine if a limiting distribution exists and what its values are, werewrite the forward Kolmogorov differential equations in Eq. (4),

dPi0(t)dt

= −λ0Pi0(t) + µ1Pi1(t) ,

dPi j(t)dt

= λ j−1Pi, j−1(t) − (λ j + µ j)Pi j(t) + µ j+1Pi, j+1(t) , j ≥ 1 ,

with initial conditions Pi j(0) = δi j. If we take the limit as t → ∞ to these equations, the limit ofthe right hand sides exists according to Eq. (8). The limit on the left hand side, the derivativesP′i j(t), also exists. Since the probabilities are converging to a constant, the limit of these deriva-tives must be zero [9]. These equations can be solved recursively. For the first equation, or whenj = 0:

limt→∞

dPi0(t)dt

= −λ0 limt→∞

Pi0(t) + µ1 limt→∞

Pi1(t)

0 = −λ0π0 + µ1π1 ,

orπ1 =

λ0

µ1π0 .

From the second equation,

limt→∞

dPi j(t)dt

= λ j−1 limt→∞

Pi, j−1(t) − (λ j + µ j) limt→∞

Pi j(t) + µ j+1 limt→∞

Pi, j+1(t)

0 = λ j−1π j−1 − (λ j + µ j)π j + µ j+1π j+1 ,

when j = 1:

π2 =λ1

µ2π1 =

λ0λ1

µ1µ2π0 ,

when j = 2:

π3 =λ2

µ3π2 =

λ0λ1λ2

µ1µ2µ3π0 ,

and so on. Thus,

πi+1 =λi

µi+1πi . (9)

9



Or, applying the induction hypothesis, the stationary probability distribution equals

πi =λ0λ1 · · · λi−1

µ1µ2 · · · µiπ0 , i = 1, 2, 3, · · · (10)

Then

µi+1πi+1 = (λi + µi)πi − λi−1πi−1

=

(λ0λ1 · · · λi−1(λi + µi)

µ1µ2 · · · µi−λ0λ1 · · · λi−1

µ1µ2 · · · µi−1

)π0

=λ0λ1 · · · λi−1

µ1µ2 · · · µi−1

(λ1 + µi

µi− 1

)π0 ,

or,

πi+1 =λ0λ1 · · · λi

µ1µ2 · · · µi+1π0 . (11)

Thus, if the state space is infinite, {0, 1, 2, · · · }, a unique positive stationary probability distribu-tion π for a general birth and death process exists, with

µi > 0 and λi−1 > 0 for i = 1, 2, 3, · · ·

Since∞∑

i=0πi = 1, which we expand

∞∑i=0

πi = 1

π0 +

∞∑i=1

πi = 1

π0

1 +

∞∑i=1

πi

π0

= 1 ,

from which we solve for π0 by invoking Eq. (10),

π0 =1

1 +∞∑

i=1

λ0λ1 · · · λi−1

µ1µ2 · · · µi

. (12)

10



If the state space is finite, {0, 1, 2, · · · ,N}, then a unique positive stationary probability distribu-tion π exists if and only if

µi > 0 and λi−1 > 0 for i = 1, 2, · · · ,N .

The stationary probability distribution is given by Eqs. (10) and (12), where the index and thesummation extend from i = 1, 2, · · · ,N.

2.3 Pure Birth

In a pure birth process, the chance of an event occurring at a given instant of time dependsupon the number of events that have already occurred. An example of this phenomenon is thereproduction of living organisms, hence the name of the process. This process considers only thebirth or arrival of new members, permitted with certain conditions, such as, sufficient food, nomortality, and no migration out of the system [9].

Let X(t) represent the number of births in the time interval (0, t]. We define a pure birth processas a Markov process satisfying the following:


=

λi∆t + o(∆t) , j = 1

1 − λi∆t + o(∆t) , j = 0

0 , j < 0 .

(13)

By analyzing the possibilities at time t just prior to time t + ∆t, for ∆t is sufficiently small, wederive a system of Kolmogorov’s forward differential equations for a pure birth process. Theprobabilities Pn(t) = Prob{X(t) = n} are solutions of the differential equations

dP0(t)dt

= −λ0P0(t) ,

dPn(t)dt

= −λnPn(t) + λn−1Pn−1(t) ,(14)

with initial conditionsP0(0) = 1 , Pn(0) = 0 , n > 0 .

The first equation of Eq. (14) can be solved immediately. With P0(0) = 1, then

P0(t) = e−λ0t , for t > 0 .

For the second equation of Eq. (14), there are several ways to compute the solutions:

11



(1) To solve the differential equations recursively, introduce

Rn(t) = eλntPn(t) , for n = 0, 1, 2, · · ·

and rearrange the second equation of Eq. (14) to get

λn−1Pn−1(t) =dPn(t)

dt+ λnPn(t) ,

which is used for the following. Differentiating Rn with respect to t yields

dRn(t)dt

= λneλntPn(t) + eλnt dPn(t)dt

= eλnt[λnPn(t) +

Pn(t)dt

]= eλntλn−1Pn−1(t) .

Integrating both sides of these equations and using the initial conditionRn(0) = 0 gives

Rn(t) =

∫ t

0eλn xλn−1Pn−1(x)dx , for n ≥ 1 .

Substituting Rn(t) on the left hand side,

eλntPn(t) =

∫ t

0eλn xλn−1Pn−1(x)dx ,

or

Pn(t) = e−λnt∫ t

0eλn xλn−1Pn−1(x)dx

= λn−1e−λnt∫ t

0eλn xPn−1(x)dx , n = 1, 2, · · · (15)

It can be seen that all Pk(t) ≥ 0, but there is still a possibility that [9]

∞∑n=0

Pn(t) < 1 .

To ensure that∑∞

n=0 Pn(t) = 1 for all t, we must restrict the λk according to the following[9]

∞∑n=0

Pn(t) = 1 if and only if∞∑

n=0

1λn

= ∞ .

12



The integral equation (15) is solved as follows

n = 1 : P1(t) = λ0e−λ1t

t∫0

eλ1 xP0(x)dx

= λ0

(e−λ0t

λ1 − λ0+

e−λ1t

λ0 − λ1

),

n = 2 : P2(t) = λ1e−λ2t

t∫0

eλ2 xP1(x)dx

= λ0λ1

(e−λ0t

(λ1 − λ0)(λ2 − λ0)+

e−λ1t

(λ0 − λ1)(λ2 − λ1)+

e−λ2t

(λ0 − λ2)(λ1 − λ2)

).

Continuing the computations, we obtain that for n > 1 and initial conditionPn(0) = 0,

Pn(t) = Prob{X(t) = n | X(0) = 0}

= λ0 · · · λn−1

(B0,ne−λ0t + · · · + Bn,ne−λnt

),

(16)

whereB0,n =

1(λ1 − λ0)(λ2 − λ0) · · · (λn − λ0)

Bk,n =1

(λ0 − λk) · · · (λk−1 − λk)(λk+1 − λk) · · · (λn − λk)

Bn,n =1

(λ0 − λn)(λ1 − λn) · · · (λn−1 − λn).

(17)

(2) Alternatively, we can also explicitly determine Pi j(t) in the case of a pure birth process, byletting Xk denote the time the process spends in state k before making a transition into statek + 1 for k ≥ 1. Suppose that the process is presently in state i, and let state j > i. ThenXi is the time it spends in state i before moving to state i + 1, and Xi+1 is the time it thenspends in state i + 1 before moving to state i + 2, and so on. It follows that

∑ j−1k=i Xk is the

time it takes until the process enters state j. If the process has not yet entered state j bytime t, then its state at time t is smaller than j, and vice versa [7], that is

X(t) < j if and only if Xi + · · · + X j−1 > t .

Therefore, for i < j, we have for a pure birth process

Pi j(t) = Prob{X(t) < j | X(0) = i} = Prob

j−1∑k=i

Xk > t

.

13



Since the random variables are hyperexponentially distributed (see hyperexponential dis-tribution and convolutions of distribution in Lecture 4a on properties of exponential distri-bution), we obtain the tail distribution function of

∑ j−1k=i Xk, that [7]

Prob{X(t) < j | X(0) = i} =

j−1∑k=i

e−λktj−1∏

r,k, r=i

λr

λr − λk.

Replacing j by j + 1 in the preceding gives

Prob{X(t) < j + 1 | X(0) = i} =

j∑k=i

e−λktj∏

r,k, r=i

λr

λr − λk.

Since

Prob{X(t) = j | X(0) = i} = Prob{X(t) < j + 1 | X(0) = i} − Prob{X(t) < j | X(0) = i} ,

and since Pii(t) = Prob{Xi > t} = e−λit, we have for a pure birth process having λi , λ j

when i , j [7]

Pi j(t) =

j∑k=i

e−λktj∏

r,k, r=i

λr

λr − λk−

j−1∑k=i

e−λktj−1∏

r,k, r=i

λr

λr − λk, i < j

Pii(t) = e−λit .

(18)

2.4 The Yule Process

The Yule process is an example of a pure birth process that commonly arises in physics andbiology. Let X(t) represent the population size at time t and the initial population is X(0) = N,so that Pi(0) = δiN . The population size can only increase in size because there are only births.The transition probabilities for sufficiently small ∆t are


=

1 − iλ∆t + o(∆t) j = 0

iλ∆t + o(∆t) j = 1

o(∆t) j ≥ 2

0 j < 0 .

(19)

14



Converting these transition probabilities into differential equationsfor Pi(t) = Prob{X(t) = i} gves

Pi(t + ∆t) = Prob{X(t + ∆t) = i}

= Prob{X(t) = i, X(t + ∆t) − X(t) = 0}+ Prob{X(t) = i − 1, X(t + ∆t) − X(t) = 1}+ Prob{X(t) = i − 2, X(t + ∆t) − X(t) ≥ 2}

= Prob{X(t) = i} · Prob{X(t + ∆t) − X(t) = 0}+ Prob{X(t) = i − 1} · Prob{X(t + ∆t) − X(t) = 1}+ Prob{X(t) = i − 2} · Prob{X(t + ∆t) − X(t) ≥ 2}

= Pi(t) ·[1 − iλ∆t + o(∆t)

]+ Pi−1(t) ·

[(i − 1)λ∆t + o(∆t)

]+ Pi−2(t) · o(∆t) .

Rearranging and equating terms with o(∆t) yield

Pi(t + ∆t) − Pi(t) = (i − 1)λ∆tPi−1(t) − iλ∆tPi(t) + o(∆t) .

Dividing both sides by ∆t and taking the limit as ∆ → 0 lead to the system of forward Kol-mogorov equations

dPi(t)dt

= (i − 1)λPi−1(t) − iλPi(t) , i = N,N + 1,N + 2, · · ·

dPi(t)dt

= 0 , i = 0, 1, 2, · · · ,N − 1

(20)

Its solution is given byPi(t) = e−λt(1 − e−λt)i−1 , i ≥ 1 . (21)

Several ways of solving the differential-difference equation of the Yule process Eq. (20) are asfollows:

(1) Using the solution of the recursive method of Eq. (16):

The general solution is analogous to Eq. (16), but for pure birth processes starting fromX(0) = 1, [9]

Pi(t) = λ1 · · · λi−1

(B1,ie−λ1t + · · · + Bi,ie−λit

), i > 1 . (22)

15



When λi = iλ,

B1,i =1

(λ2 − λ1)(λ3 − λ1)(λ4 − λ1) · · · (λi − λ1)

=1

(2λ − λ)(3λ − λ)(4λ − λ) · · · (iλ − λ)

=1

λ(1)λ(2)λ(3) · · · λ(i − 1)

=1

λi−1(1)(2)(3) · · · (i − 1)

=1

λi−1(i − 1)!,

and

B2,i =1

(λ1 − λ2)(λ3 − λ2) · · · (λi − λ2)

=1

λi−1(−1)(1)(2) · · · (i − 2)

=−1

λi−1(i − 2)!,

so, in general,

Bk,i =1

(λ1 − λk) · · · (λk−1 − λk)(λk+1 − λk) · · · (λi − λk)

=(−1)k−1

λi−1(k − 1)!(i − k)!.

Thus the solution is

Pi(t) = λi−1(i − 1)!(B1,ie−λt + B2,ie−2λt + · · · + Bi,ie−iλt

)=

i∑k=1

(i − 1)!(k − 1)!(i − k)!

(−1)k−1e−kλt

= e−λtn−1∑j=0

(i − 1)!j!(i − 1 − j)!

(e−λt) j

= e−λt(1 − e−λt

)i−1, i ≥ 1 . (23)

16



(2) Alternatively, using Eq. (18), for the case i = 1:

P1 j(t) = Prob{X(t) ≤ j | X(0) = 1}

=

j∑k=1

e−kλtj∏

r,k, r=1

rr − k

−

j−1∑k=1

e−kλtj−1∏

r,k, r=1

rr − k

= e− jλtj−1∏r=1

rr − j

+

j−1∑k=1

e−kλt

j∏r,k, r=1

rr − k

−

j−1∏r,k, r=1

rr − k

= e− jλt(−1) j−1 +

j−1∑k=1

e−kλt

(j

j − k− 1

) j−1∏r,k, r=1

rr − k

.

Now,

kj − k

j−1∏r,k, r=1

rr − k

=( j − 1)!

(1 − k)(2 − k) · · · (k − 1 − k)( j − k)!

= (−1)k−1(

j − 1k − 1

).

Hence,

P1 j(t) =

j∑k=1

(j − 1k − 1

)e−kλt(−1)k−1

= e−λtj−1∑i=0

(j − 1

i

)e−iλt(−1)i

= e−λt(1 − e−λt) j−1 , j ≥ 1 , (24)

which is exactly the same as the solution in Eq. (23).

The solution in Eq. (24) states that starting with one individual, the population size at time thas a geometric distribution with mean eλt . If the population starts with i individuals, thenwe can regard each of these individuals as starting their own independent Yule process,and so the population at time t will be the sum of i independent and identically distributedgeometric random variables with parameter e−λt. But this means that the conditional dis-tribution of X(t), given that X(0) = i, is the same as the distribution of the number of timesthat a coin that lands heads on each flip with probability e−λt must be flipped to amass atotal of i heads. Hence, the population size at time t has a negative binomial distributionwith parameters i and e−λt, thus

Pi j(t) =

(j − 1i − 1

)e−iλt (1 − e−λt) j−i , j ≥ i ≥ 1 . (25)

17



(3) Using the probability generating function (p.g.f.):

Observe that the state space for this process is {1, 2, 3, · · · }. All of the states are transientand the transition probabilities Pn(t) all depend on time t or non-stationary probabilitydistribution.

Recall that the p.g.f., denoted �(t), is given by

�(z) =

∞∑i=0

Pi zi .

For the sake of generality, here we alter the initial conditions where now PN(0) = 1 andPi(0) = 0. We derive the partial differential equation for the p.g.f. by multiplying thedifferential equations by zi and sum over i,

∂�(z, t)∂t

=

∞∑i=N+1

(N + 1 − 1)λPN+1−1 zN+1 −

∞∑i=N

NλPN zN

=

∞∑i=N+1

NλPN zN−1z2 −

∞∑i=N

NλPN zN−1z

= λz2∞∑

i=N

iPi zi−1 − λz∞∑

i=N

iPi zi−1

= λz(z − 1)∞∑

i=N

iPi zi−1 .

Recall that the first derivative of � with respect to z is

∂�

∂z=

∞∑i=1

iPi zi−1 ,

then∂�(z, t)∂t

= λz(z − 1)∂�

∂z.

The initial condition is �(z, 0) = zN .

The partial differential equation for the moment generating function (m.g.f.) can be derivedby a change of variable. Let z = eθ, where dθ/dz = 1/eθ = 1/z. There exists a closerelationship between the probability generating function �X(·) and the moment generatingfunction�X(·), that is

�(θ) = E[eθX] = �(eθ) .

Then �(eθ, t) =�(θ, t). Following the chain rule,

∂�

∂z=∂�

∂θ

dθdz

=1z∂�

∂θ.

18



The m.g.f. is a solution of the following partial differential equation

∂�

∂t= λ(eθ − 1)

∂�

∂θ,

with initial condition�(θ, 0) = eNθ.

The method of characteristics is applied to find the solution of�(θ, t). Rewrite the differ-ential equation for�(θ, t)

∂�

∂t+ λ(1 − eθ)

∂�

∂θ= 0 .

The solution is found by introducing the characteristic equations

dθdτ

= λ(1 − eθ) ,dtdτ

= 1 , anddvdτ

= 0 ,

with initial conditions

θ(s, 0) = s , t(s, 0) = 0 , and v(s, 0) = eNs .

For the first characteristic equation, separation of variables and simplification lead to

dθ1 − eθ

= λdτ ande−θdθ

e−θ − 1= λdτ .

By using a substitution u = e−θ − 1, from which du = −e−θdθ, we integrate both sides of

−duu

= λdτ orduu

= −λdτ

which results in a general solution

ln(u) = −λτ + c1 or ln(e−θ − 1) = −λτ + c1 .

The general solution can also be written

e−θ − 1 = e−λτ+c1 = C0 e−λτ .

Applying the initial condition θ(s, 0) = s solves for the constant C0 = e−s − 1. Thus theparticular solution of the first characteristic equation is

e−θ − 1 = (e−s − 1)e−λτ .

The second characteristic equation can be immediately integrated

t = τ + c2 ,

19



and using the initial condition t(s, 0) = 0, the solution is simply

t = τ .

And the solution of the third characteristic equation

v(s, τ) = eNs .

Letting v(s, τ) =�(s, τ), we can write for the m.g.f.

�(s, τ) = eNs .

The solution� must be expressed in terms of the original variables θ and t. We do this byexpressing e−s in terms of θ and t, that is

e−s = 1 − eλτ(1 − e−θ) .

Since eNs = (e−s)−N , the m.g.f. for the Yule process is

�(θ, t) =(1 − eλτ(1 − e−θ)

)−N.

The p.g.f. can be found directly from the m.g.f. by replacing e−θ = z−1 since eθ = z,

�(z, t) =(1 − eλτ(1 − z−1)

)−N

= zNe−λNt(ze−λt − (z − 1)

)−N

=zNe−λNt

(1 − z(1 − e−λt))N .

Letting p = e−λt and q = 1 − e−λt, the p.g.f. is

�(z, t) =(pz)N

(1 − qz)N . (26)

Taking N = 1, Eq. (26) can be solved trivially, that is, with

�(z, t) =pz

1 − qz,

20



then∂�(z)∂z

=p

(1 − qz)2

∣∣∣∣∣z=0

= p

∂2�(z)∂z2 =

2pq(1 − qz)3

∣∣∣∣∣z=0

= 2pq

∂3�(z)∂z3 =

6pq2

(1 − qz)4

∣∣∣∣∣∣z=0

= 6pq2

∂4�(z)∂z4 =

24pq3

(1 − qz)5

∣∣∣∣∣∣z=0

= 24pq3

...

and from Eq. (2),

Pi(t) =11!

p +12!

2!pq2−1 +13!

3!pq3−1 +14!

4!pq4−1 ,

or

Pi(t) =

(i − 1i − N

)pNqi−N , for N = 1 and i = 1, 2, 3, · · ·

In general, for N ≥ 1, the p.g.f. and m.g.f. for the Yule process correspond to a negativebinomial distribution. From the binomial coefficient, i must be greater than or equal to N,or i ≥ N. Then taking i = i + N and replacing p by e−λt and q by 1 − e−λt, the probabilitiesPi(t) can also be written as,

Pi+N(t) =

(N + i − 1

i

)e−λNt (1 − e−λt)i , i = 0, 1, 2, · · · (27)

By letting k = i + N, another way to write the probabilities is

Pk(t) =

(k − 1k − N

)e−λNt(1 − e−λt)k−N , k = N,N + 1, · · ·

When N = 1 or assuming that the population starts with one individual, then k = i + 1 andEq. (27) becomes

Pi+1(t) = e−λt(1 − e−λt)i or Pi(t) = e−λt(1 − e−λt)i−1 . (28)

The mean and variance for the Yule process are

m(t) =Np

= N eλt and σ2(t) =Nqp2 = N e2λt(1 − e−λt) .

The probabilities with four different values of t using Eq. (27) and three sample paths of theYule process are shown in Fig. 1. Python code for these figures are provided in the PythonImplementation section 3.1.

21



Figure 1: Plots of probability distributions Pi+N(t) for the Yule process using Eq. (27) with N = 5(left) and population size over time with three sample paths (right), both when λ = 0.5 andX(0) = 5. The mean and variance are m(t) = 5e0.5t and σt(t) = 5et(1 − e−0.5t).

2.5 Pure Death

Complementing the increasing pure birth process is the pure death process that is decreasing.Since the only event in the pure death process is a death, the process decreases successivelythrough states N,N − 1, · · · , 2, 1 and ultimately is absorbed in state 0 (extinction). The puredeath process is specified by the death parameters µk > 0 for k = 1, 2, · · · ,N, where the so-journ/interevent time in state k is exponentially distributed with parameter µk.

The infinitesimal transition probabilities of the pure death process whose state space is 0, 1, 2, · · · ,Nare


=

µi∆t + o(∆t) , j = −1

1 − µi∆t + o(∆t) , j = 0

0 , j > 0

(29)

for i = 1, 2, · · · ,N for the first and second equations and i = 0, 1, · · · ,N for the third equation.As we have previously discussed in the general birth and death process, it is common to assignµ0 = 0.

If the death parameters µ1, µ2, · · · , µN are distinct, that is µi , µ j if i , j, taking from Eq. (4), theforward Kolmogorov equation for the pure death process at i = N is given by

dPN(t)dt

= −µN PN(t) .

If the initial condition is X(0) = N, or PN(0) = Prob{X(0) = N} = 1, thus the solution, or theexplicit transition probabilities are

PN(t) = e−µN t .

22



For i < N,dPi(t)

dt= µi+1Pi+1(t) − µiPi(t) ,

and the solutions are given by

Pi(t) = Prob{X(t) = i | X(0) = N}

= µi+1µi+2 · · · µN(Ai,ie−µit + · · · + AN,ie−µN t) (30)

whereAk,i =

1(µN − µk) · · · (µk+1 − µk)(µk−1 − µk) · · · (µi − µk)

.

2.6 Linear Death

As the pure death process complements the pure birth process, the linear death process comple-ments the Yule process. In the linear death process, or, also called the simple death process, thedeath rates are proportional to population size, µi = iµ. The infinitesimal transition probabilitiesof the linear death process are


=

iµ∆t + o(∆t) , j = −1

1 − iµ∆t + o(∆t) , j = 0

o(∆t) , j ≤ −2

0 , j > 0

(31)

Since the process begins in state N, the state space is {0, 1, 2, · · · ,N}.

Converting these transition probabilities into differential equations

23



for Pi(t) = Prob{X(t) = i} gives

Pi(t + ∆t) = Prob{X(t + ∆t) = i}

= Prob{X(t) = i, X(t + ∆t) − X(t) = 0}+ Prob{X(t) = i + 1, X(t + ∆t) − X(t) = −1}+ Prob{X(t) = i + 2, X(t + ∆t) − X(t) ≤ −2}

= Prob{X(t) = i} · Prob{X(t + ∆t) − X(t) = 0}+ Prob{X(t) = i + 1} · Prob{X(t + ∆t) − X(t) = −1}+ Prob{X(t) = i + 2} · Prob{X(t + ∆t) − X(t) ≤ −2}

= Pi(t) ·[1 − iµ∆t + o(∆t)

]+ Pi+1(t) ·

[(i + 1)µ∆t + o(∆t)

]+ Pi+2(t) · o(∆t) .

Rearranging and equating terms with o(∆t) yield

Pi(t + ∆t) − Pi(t) = (i + 1)µ∆tPi+1(t) − iµ∆tPi(t) + o(∆t) .

Dividing both sides by ∆t and taking the limit as ∆ → 0 lead to the system of forward Kol-mogorov equations

dPi(t)dt

= (i + 1)µPi+1(t) − iµPi(t)

dPN(t)dt

= −NµPN(t) ,

for i = 0, 1, 2, · · · ,N − 1 and with initial conditions PN(0) = 1 and Pi(0) = 0 for i , N.

As in the Yule process, the probability distribution may be found by using the probability generat-ing function and method of characteristics. First, we multiply the forward Kolmogorov equations

24



by zi and sum over i,

∂�(z, t)∂t

=

∞∑i=N−1

(i + 1)µPi+1(t)zi −

∞∑i=N

iµPi(t)zi

=

∞∑i=N−1

(N − 1 + 1)µPN−1+1(t)zN−1 −

∞∑i=N

NµPN(t)zN

=

∞∑i=N

NµPN(t)zN−1 −

∞∑i=N

NµPN(t)zN−1z

= µ

∞∑i=N

iPi(t)zi−1 − µz∞∑

i=N

iPi(t)zi−1

= µ(1 − z)∞∑

i=N

iPi(t)zi−1

to get the partial differential equation for the p.g.f.,

∂�(z, t)∂t

= µ(1 − z)∂�

∂z, �(z, 0) = zN .

Substituting z = eθ, the m.g.f. is

∂�

∂t= µ(e−θ − 1)

∂�

∂θ, �(θ, 0) = eNθ .

After applying the method of characteristics, we obtain the solutions

�(z, t) =(1 − e−µt(1 − z)

)N=

(1 − e−µt + e−µtz

)N

and�(θ, t) =

(1 − e−µt(1 − eθ)

)N.

By letting p = e−µt and q = 1 − p = 1 − e−µt, the p.g.f. is

�(z, t) = (q + pz)N ,

which corresponds to a binomial distribution b(N, p). Since

Pi(t) =1i!

∂i�

∂zi

∣∣∣∣∣∣z=0

,

25



from which we show that,

k = 0 → P0(t) = qN

k = 1 → P1(t) =11!

∂�

∂z

∣∣∣∣∣z=0

=11!

NqN−1 p

k = 2 → P2(t) =12!

∂2�

∂z2

∣∣∣∣∣∣z=0

=12!

N(N − 1) qN−2 p2

k = 3 → P3(t) =13!

∂3�

∂z3

∣∣∣∣∣∣z=0

=13!

N(N − 1)(N − 2) qN−3 p3

...

thus, the probabilities are

Pi(t) =N!

i!(N − i)!piqN−i =

(Ni

)e−iµt (1 − e−µt)N−i

, (32)

for i = 0, 1, 2, · · · ,N. The mean is obtained by first differentiating the p.g.f. with respect to z,that is

d�dz

= N(q + pz)N−1 p ,

and the mean, now denoted by m(t), is

m(t) = �′(1) = N p ,

or, by p = e−µt

m(t) = Ne−µt ,

which corresponds to exponential decay. Whereas, the variance is

σ2(t) = �′′

(1) + �′

(1) −(�′

(1))2

= Ne−µt (1 − e−µt) .Example 4b.1: Linear Death Process

Taken from Example 5.13 in [1]:

In a linear death process, the only event is a death, that is, state i → i − 1. For ∆t sufficiently

26



small, the transition probabilities are


=

di∆t + o(∆t) j = −1

1 − di∆t + o(∆t) j = 0

o(∆t) j ≤ − 2

0 j > 0 .

Given X(Wi) = n, then α(n) = dn. Therefore, the interevent time Ti is

Ti = −ln(U)

dn.

The next event is a death n→ n − 1, X(Wi+1) = n − 1. The deterministic analogue of this simpledeath process is the differential equation dn/dt = −dn, with initial condition n(0) = N, and thesolution

n(t) = N e−dt . �

The probability distributions and population size over time with three sample paths of the lineardeath process are shown in Fig. 2. The Python code to produce these results are given in thePython Implementation section 3.2. The code are also available on the course website.

Figure 2: Plots of probability distributions Pk(t) for the linear death process using Eq. (32) (left)and population size over time with three sample paths (right), both when µ = 0.5 and X(0) = 20.The mean and variance are m(t) = 20e−0.5t and σ2(t) = 20e−0.5t(1 − e−0.5t).

2.7 Simple Birth and Death

In a simple birth and death process, X(t) is allowed to increase and decrease. If at time t theprocess is in state n, after a random sojourn time, it may move to either of the neighboring states

27



n + 1 or n − 1. The birth and death process is irreducible if all the states communicate, and thishappens if and only if all the λn (n ≥ 0) and all the µn (n ≥ 1) are positive.

If we let X(t) denote the state of the chain at time t, then


=

1 − i(µ + λ)∆t + o(∆t) j = 0

iλ∆t + o(∆t) j = 1

iµ∆t + o(∆t) j = −1

o(∆t) j , − 1, 0, 1 .

(33)

Simple birth and death processes are also known as birth and death processes with absorbingstates. For these processes, the zero state is an absorbing state, where when the population sizebecomes zero, it remains zero thereafter.

As for the formulations in the Poisson process, we can convert the transition probabilities inEq. (35) into differential equations for Pi(t) = Prob{X(t) = i}, hence

Pi(t + ∆t) = Prob{X(t + ∆t) = i}

= Prob{X(t) = i, X(t + ∆t) − X(t) = 0}+ Prob{X(t) = i − 1, X(t + ∆t) − X(t) = 1}+ Prob{X(t) = i + 1, X(t + ∆t) − X(t) = −1}

= Prob{X(t) = i} · Prob{X(t + ∆t) = X(t)}+ Prob{X(t) = i − 1} · Prob{X(t + ∆t) − X(t) = 1}+ Prob{X(t) = i + 1} · Prob{X(t + ∆t) − X(t) = −1}

= Pi(t) ·[1 − i(µ + λ)∆t + o(∆t)

]+ Pi−1(t) ·

[(i − 1)λ∆t + o(∆t)

]+ Pi+1(t) ·

[(i + 1)µ∆t + o(∆t)

].

Rearranging and equating terms with o(∆t) give us

Pi(t + ∆t) − Pi(t) = (i + 1)µ∆tPi+1(t) + (i − 1)λ∆tPi−1(t) − i(µ + λ)∆tPi(t) + o(∆t) .

Dividing both sides by ∆t and taking the limit as ∆t → 0 lead to the following system of

28



differential-difference equations

dPi(t)dt

= (i − 1)λPi−1(t) + (i + 1)µPi+1(t) − i(µ + λ)Pi(t) ,

dP0(t)dt

= µP1(t)

(34)

for i = 1, 2, · · · with initial conditions Pi(0) = δiN . Multiplying the first equation by zi andsumming over i we obtain

∂�(z, t)∂t

=

∞∑i=N+1

(N + 1 − 1)λPN+1−1(t)zN+1 +

∞∑i=N−1

(N − 1 + 1)µPN−1+1(t)zN−1

−

∞∑i=N

N(µ + λ)PN(t)zN

=

∞∑i=N+1

NλPN(t)zN−1z2 +

∞∑i=N−1

NµPN(t)zN−1 −

∞∑i=N

N(µ + λ)PN(t)zN−1z

= λz2∞∑

i=N

iPi(t)zi−1 + µ

∞∑i=N

iPi(t)zi−1 − z(µ + λ)∞∑

i=N

iPi(t)zi−1

=[λz2 − (µ + λ)z + µ

] ∞∑i=N

iPi(t)zi−1 .

The p.g.f. is a solution of

∂�(z, t)∂t

=[µ(1 − z) + λz(z − 1)

]∂�∂z

, �(z, 0) = zN

and the m.g.f. is a solution is

∂�

∂t=

[µ(e−θ − 1) + λ(eθ − 1)

]∂�∂θ

, �(θ, 0) = eθN .

The characteristic equations are

dtdτ

= 1 ,

dθdτ

= −[µ(e−θ − 1) + λ(eθ − 1)

],

d�dτ

= 0 ,

29



after immediately letting v(s, τ) =�(s, τ). The initial conditions are

t(s, 0) = 0 , θ(s, 0) = s , and �(s, 0) = eNs .

The solutions satisfyt = τ and �(s, τ) = eNs .

We solve for τ in the following ways

• If µ = λ:By a change of variable x = eθ, the second characteristic equation becomes

dθλ(e−θ − 1) + λ(eθ − 1)

= −dτ

dθλ(eθ + e−θ) − 2λ

= −dτ

dxx[λ(x + x−1) − 2λ

] = −dτ

dxλ(x2 − 2x + 1)

= −dτ

dxλ(x − 1)2 = −dτ

Using a substitution u = x − 1, we integrate both sides

1λ

∫duu2 = −

∫dτ

−1λ u

= −τ + c1 ,

orτ =

1λ u

+ c1 =1

λ(eθ − 1)+ c1 ,

where c1 is the arbitrary constant of integration. Applying the initial condition θ(s, 0) = s,we obtain

0 =1

λ(es − 1)+ c1 ,

orc1 = −

1λ(es − 1)

.

Hence,

τ =1

λ(eθ − 1)−

1λ(es − 1)

.

30



• If µ , λ:Similar way as the above,

dθµ(e−θ − 1) + λ(eθ − 1)

= −dτ

dxx[µ(x−1 − 1) + λ(x − 1)

] = −dτ

dxµ(1 − x) + λ(x2 − x)

= −dτ

dxλx2 − (λ + µ)x + µ

= −dτ .

The left hand side equation can be resolved into partial fractions as follows

1λx2 − (λ + µ)x + µ

≡1

(λx − µ)(x − 1)

≡A

λx − µ+

Bx − 1

,

where A and B are constants to be determined by algebraic addition, i.e.,

1(λx − µ)(x − 1)

≡A(x − 1) + B(λx − µ)

(λx − µ)(x − 1).

Since the denominators are the same on each side of the identity, then the numerators areequal to each other. Thus,

1 ≡ A(x − 1) + B(λx − µ) .

The constants A and B are determined by choosing the values of x to make the term in Aor B equal to zero. That is, when x = 1, then

1 ≡ A(0) + B(λ − µ)

B =1

λ − µ,

and when x = µ/λ, then

1 ≡ A(µ

λ− 1) + B(0)

A =λ

µ − λ.

The characteristic equation becomes(λ

µ − λ·

1λx − µ

+1

λ − µ·

1x − 1

)dx = −dτ .

31



Integrating both sides and using substitutions

λ

µ − λ

∫dx

λx − µ+

1λ − µ

∫dx

x − 1= −

∫dτ

1µ − λ

ln u +1

λ − µln v = −τ + c2

−1

µ − λln

(vu

)= −τ + c2

where u = λx − µ and v = x − 1. Thus,

τ =1

µ − λln

(eθ − 1λeθ − µ

)+ c2 .

Applying the initial condition θ(s, 0) = s gives

c2 = −1

µ − λln

(es − 1λes − µ

),

and the particular solution of the characteristic equation is

τ =1

µ − λln

(eθ − 1λeθ − µ

)−

1µ − λ

ln(

es − 1λes − µ

)

=1

µ − λ

[ln

(eθ − 1)(λes − µ)(λeθ − µ)(es − 1)

].

Because�(s, τ) = (es)N , these relations are solved for es,

es =

eτ(µ−λ)(λeθ − µ) − µ(eθ − 1)eτ(µ−λ)(λeθ − µ) − λ(eθ − 1)

, if µ , λ

1 − (λτ − 1)(eθ − 1)1 − λτ(eθ − 1)

, if µ = λ .

The m.g.f. now can be expressed in terms of θ and t, that is

�(θ, t) =

(et(µ−λ)(λeθ − µ) − µ(eθ − 1)et(µ−λ)(λeθ − µ) − λ(eθ − 1)

)N

, if µ , λ

(1 − (λt − 1)(eθ − 1)

1 − λt(eθ − 1)

)N

, if µ = λ .

32



Making the change of variable θ = ln(z), we obtain the p.g.f.,

�(z, t) =

(et(µ−λ)(λz − µ) − µ(z − 1)et(µ−λ)(λz − µ) − λ(z − 1)

)N

, if µ , λ

(1 − (λt − 1)(z − 1)

1 − λt(z − 1)

)N

, if µ = λ .

Obtaining the probabilities Pi(t) is not as straightforward as it was for the Yule and linear deathprocesses. We follow calculation results in [1]. The first term in the expansion of �(z, t) isP0(t) = �(0, t):

P0(t) =

(µ − µe(µ−λ)t

λ − µe(µ−λ)t

)N

, if µ , λ

(λt

1 + λt

)N

, if µ = λ .

The probability of extinction, P0(t), has a simple expression when t → ∞,

P0(∞) = limt→∞

P0(t) =

1 , if λ ≤ µ(µ

λ

)Nif λ > µ .

If the probability of death is greater than or equal to the probability of birth, or λ ≤ µ, then in thelong run as t → ∞ the probability of losing all of the initial population N approaches one. Butif the probability of birth is greater than the probability of death, or λ > µ, then as t → ∞ theprobability of losing all of the initial population is (µ/λ)N . The result when λ > µ as t → ∞ isreminiscent of a semi-infinite random walk with an absorbing barrier at x = 0, or the gambler’sruin problem, where the probability of losing a game is µ and the probability of winning a gameis λ [1].

The mean m(t) and variance σ2(t) of the simple birth and death process are:

• For µ , λ

m(t) = Ne(λ−µ)t and σ2(t) = N(λ + µ)(λ − µ)

e(λ−µ)t(e(λ−µ)t − 1) ,

where the mean corresponds to exponential growth when λ > µ and exponential decaywhen λ < µ.

• For µ = λm(t) = N and σ2(t) = 2Nλt .

33



The population over time plots of the simple birth and death process are shown in Fig. 3. ThePython code to produce these results are given in the Python Implementation section 3.3. Thecode are also available on the course website.

Figure 3: Plots of three sample paths for the simple birth and death process when µ = 1.0,X(0) = 50, and λ = 0.1 (left) and λ = 1.0 (right). In the case λ << µ (left figure), the populationsize becomes zero and it remains in zero forever.

2.8 Simple Birth and Death Process with Immigration

A birth and death process is called a linear growth process if λi = iλ + β and µi = iµ withλ > 0, µ > 0, and β > 0. If the state i describes the current population size, then the averageinstantaneous rate of growth is iλ + β, where the factor iλ represents the natural growth of thepopulation owing to its current size and the factor β may be interpreted as the infinitesimal rateof increase of the population due to an external source, such as immigration. Note that theimmigration rate does not depend linearly on i, which looks like the Poisson process. Similarly,the probability of the state of the process decreasing by one after the elapse of a small duration oftime ∆t is iµ∆t + o(∆t), where the factor iµ gives the mean infinitesimal death rate of the presentpopulation.

For the general case of a simple birth and death process with immigration and emigration wherethe rates are λi = iλ + β and µi = iµ + ρ, state transitions are a combination of:

• birth: i −→ i + 1 at rate iλ

• death: i −→ i − 1 at rate iµ

• immigration: i −→ i + 1 at rate β

• emigration: i −→ i − 1 at rate ρ

34



The infinitesimal transition probabilities for the simple birth and death process with immigrationsare


=

1 −[β + i(µ + λ)

]∆t + o(∆t) j = 0

(β + iλ)∆t + o(∆t) j = 1

iµ∆t + o(∆t) j = −1

o(∆t) j , − 1, 0, 1 .

(35)

Because of the immigration term, the value of λ0 = β > 0.

The forward Kolmogorov differential equations for the process are

• For i = 0, where λ0 = β and µ1 = µ,

dP0(t)dt

= −βP0(t) + µP1(t)

• For i ≥ 1, where λi−1 = (i − 1)λ + β, λi = iλ + β, µi+1 = (i + 1)µ, and µi = iµ,

dPi(t)dt

=[λ(i − 1) + β

]Pi−1(t) −

[(λ + µ)i + β

]Pi(t) + µ(i + 1)Pi+1(t) .

Letting X(0) = N, for the second equation with i , 1, we apply the generating function techniqueby multiplying the equation by zi and summing over i,

∂�

∂t=

∞∑i=N+1

[λ(i − 1) + β

]Pi−1(t)zi −

∞∑i=N

[iλ + iµ + β

]Pi(t)zi +

∞∑i=N−1

µ(i + 1)Pi+1(t)zi

= λz2∞∑

i=N

iPi(t)zi−1 + βz∞∑

i=N

Pi(t)zi − (λ + µ)z∞∑

i=N

iPi(t)zi−1 − β

∞∑i=N

Pi(t)zi

+ µ

∞∑i=N

iPi(t)zi−1

=[λz(z − 1) + µ(1 − z)

] ∞∑i=N

iPi(t)zi−1 + β(z − 1)∞∑

i=N

Pi(t)zi ,

we obtain the p.g.f.,∂�

∂t=

[λz(z − 1) + µ(1 − z)

]∂�∂z

+ β(z − 1)� ,

35



and subsequently, the m.g.f. [1],

∂�

∂t=

[λ(eθ − 1) + µ(e−θ − 1)

]∂�∂θ

+ β(eθ − 1)� ,

with initial condition�(θ, 0) = eNθ. The solution of the m.g.f [1] is

�(θ, t) =(λ − µ)β/λ

[µ(ρ − 1) − eθ(µρ − λ)

]N[(λρ − µ) − λeθ(ρ − 1)

]N+β/λ,

and the p.g.f.,

�(z, t) =(λ − µ)β/λ

[µ(ρ − 1) − z(µρ − λ)

]N[λρ − µ − λz(ρ − 1)

]N+β/λ,

where ρ = e(λ−µ)t, X(0) = N, and λ , µ.

The mean can be calculated by differentiating�(θ, t) with respect to θ and evaluating at θ = 0,

m(t) =

ρ(Nµ − Nλ − β) + β

µ − λ, if λ , µ

βt + N , if λ = µ .

(36)

For the cases λ > µ and λ = µ, the process is explosive, where when λ > µ, the mean increasesexponentially in time and when λ = µ, the mean increases linearly in time. However, the processis nonexplosive in the case λ < µ, where the mean approaches a constant,

m(∞) =β

µ − λ, λ < µ .

Also, for the case λ < µ, the process is irreducible and has a unique positive stationary distri-bution. These conditions of being nonexplosive, irreducible, and having a positive stationarydistribution imply that the process is positive recurrent [1]. Then, Theorem 8 in Lecture 4aimplies the limiting distribution exists and equals the (unique positive) stationary distribution.

Example 4b.1

Taken from Example 6.1 on pp. 246–247 in [1]:

A continuous-time birth and death Markov chain has a birth rate λi = b and a death rate µi = idfor i = 0, 1, 2, · · · . Find the stationary probability distribution.

36



Solution:

To find the stationary probability distribution π, we use formulations derived in subsection 2.2.

For:

λi = b ,→ then λ0λ1 · · · λi−1 = b · b · · · b = bi ,

µi = id ,→ then µ1µ2µ3 · · · µi = 1d · 2d · 3d · · · id = dii! ,

from which,λ0λ1 · · · λi−1

µ1µ2µ3 · · · µi=

bi

dii!=

(b/d)i

i!,

then, the general equation of the stationary probability distribution is given by

πi =(b/d)i

i!π0 . (37)

Since, according to the power series of an exponential function,

1 +

∞∑i=1

λ0λ1 · · · λi−1

µ1µ2µ3 · · · µi= 1 +

∞∑i=1

(b/d)i

i!

= 1 +(b/d)

1!+

(b/d)2

2!+

(b/d)3

3!+ · · ·

= eb/d ,

from Eq. (12),

π0 =1

1 +∞∑

i=1

λ0λ1 · · · λi−1

µ1µ2µ3 · · · µi

=1

eb/d

= e−b/d ,

and from Eq. (10), the unique stationary probability distribution is

πi =λ0λ1 · · · λi−1

µ1µ2µ3 · · · µiπ0

=(b/d)i

i!e−b/d ,

which is a Poisson distribution with parameter b/d, for i = 0, 1, 2, 3, · · · . The Markov chain ispositive recurrent for all λ and µ.

37



Example 4b.2

Taken from Example 6.3 on pp. 256 in [1]:

Consider a simple birth and death process with immigration. Let λ = 0.5 and µ = 1. Find thestationary probability distribution when β = 0.5, β = 1.0, β = 1.5, and β = 5.0.

Solution:

The death rate is µi = iµ = i, andthe average instantaneous rate of growth is λi = iλ + β = 0.5i + β,with various values of β as above.

Using the same steps as in Example 4b.1, that is

(1) When β = 0.5

∗ λi = 0.5(i + 1), then

λ0λ1 · · · λi−1 = [0.5(0 + 1)] · [0.5(1 + 1)] · [0.5(2 + 1)] · · · [0.5(i − 1 + 1)]= [0.5(1)] · [0.5(2)] · [0.5(3)] · · · [0.5(i)]= 0.5i i!

∗ µi = iµ, thenµ1µ2µ3 · · · µi = 1 · 2 · 3 · · · i = i! .

Combining the above,

πi =λ0λ1 · · · λi−1

µ1µ2 · · · µiπ0

=

(0.5(1)

1

) (0.5(2)

2

) (0.5(3)

3

)· · ·

(0.5(i)

i

)π0

=0.5i (i + 0)!

0! i!π0

=0.5i i!

i!π0

= (0.5)i π0 .

(2) When β = 1

∗ λi = 0.5i + 1,

λ0λ1 · · · λi−1 = [0.5(0) + 1] · [0.5(1) + 1] · [0.5(2) + 1] · · · [0.5(i − 1) + 1]= [0.5(2)] · [0.5(3)] · [0.5(4)] · · · [0.5(i + 1)]

38



∗ µi = iµ,µ1µ2µ3 · · · µi = 1 · 2 · 3 · · · i = i! .

Then

πi =λ0λ1 · · · λi−1

µ1µ2 · · · µiπ0

=

(0.5(2)

1

) (0.5(3)

2

) (0.5(4)

3

)· · ·

(0.5(i + 1)

i

)π0

=0.5i (i + 1)!

1! i!π0

=0.5i (i + 1)!

i!π0 .

(3) When β = 1.5

∗ λi = 0.5i + 1.5,

λ0λ1 · · · λi−1 = [0.5(0) + 1.5] · [0.5(1) + 1.5] · [0.5(2) + 1.5] · · · [0.5(i − 1) + 1.5]= [0.5(3)] · [0.5(4)] · [0.5(5)] · · · [0.5(i + 2)]

∗ µi = iµ,µ1µ2µ3 · · · µi = 1 · 2 · 3 · · · i = i! .

Then

πi =λ0λ1 · · · λi−1

µ1µ2 · · · µiπ0

=

(0.5(3)

1

) (0.5(4)

2

) (0.5(5)

3

)· · ·

(0.5(i + 2)

i

)π0

=0.5i (i + 2)!

2! i!π0 .

(4) When β = 5

∗ λi = 0.5i + 5,

λ0λ1 · · · λi−1 = [0.5(0) + 5] · [0.5(1) + 5] · [0.5(2) + 5] · · · [0.5(i − 1) + 5]= [0.5(10)] · [0.5(11)] · [0.5(12)] · · · [0.5(i + 9)]

∗ µi = iµ,µ1µ2µ3 · · · µi = 1 · 2 · 3 · · · i = i! .

39



Then

πi =λ0λ1 · · · λi−1

µ1µ2 · · · µiπ0

=

(0.5(10)

1

) (0.5(11)

2

) (0.5(12)

3

)· · ·

(0.5(i + 9)

i

)π0

=0.5i (i + 9)!

9! i!π0 .

We can deduce that, the general equation of the stationary probability distribution πi, with birthrate λ, migration rate β, and death rate µ, is given by

πi =λi (i + β/λ − 1)!(β/λ − 1)! µi i!

π0 ,

orπi = (λ/µ)i ·

(i + β/λ − 1)!i! (β/λ − 1)!

π0 . (38)

Taking β/λ = n, the term(i + n − 1)!i! (n − 1)!

=

(i + n − 1

i

),

is a binomial coefficient. Hence, we can write (38) as

πi = (λ/µ)i

(i + n − 1

i

)π0 . (39)

To calculate π0, we may use the following

π0 =1

1 +∞∑

i=1

λ0λ1 · · · λi−1

µ1µ2 · · · µi

,

(1) when β = 0.5, and n = 0.5/0.5 = 1,

1 +

∞∑i=1

λ0λ1 · · · λi−1

µ1µ2 · · · µi= 1 +

∞∑i=1

(0.5)i

= 1 + 0.5 + (0.5)2 + (0.5)3 + · · · (geometric series)

=1

1 − 0.5

= 2 ,

40



from which,

π0 =12.

The stationary distribution is a geometric distribution,

πi = (0.5)i+1 , i = 0, 1, 2, · · ·

(2) when β = 1, and n = 1/0.5 = 2,

1 +

∞∑i=1

λ0λ1 · · · λi−1

µ1µ2 · · · µi= 1 +

∞∑i=1

(0.5)i (i + 1)!i!

= 1 +(0.5) 2!

1!+

(0.5)2 3!2!

+(0.5)3 4!

3!+ · · · (binomial series)

=

∞∑i=0

(0.5)i

(2 + i − 1

i

)

=1

(1 − 0.5)2

= 4 ,

from which,

π0 =14.

The unique stationary distribution is

πi =14·

(0.5)i (i + 1)!i!

=(0.5)i+2 (i + 1)!

i!, i = 0, 1, 2, · · ·

(3) when β = 1.5, and n = 1.5/0.5 = 3,

1 +

∞∑i=1

λ0λ1 · · · λi−1

µ1µ2 · · · µi= 1 +

∞∑i=1

(0.5)i (i + 2)!2! i!

= 1 +(0.5) 3!

2! 1!+

(0.5)2 4!2! 2!

+(0.5)3 5!

2! 3!+ · · · (binomial series)

=

∞∑i=0

(0.5)i

(3 + i − 1

i

)

=1

(1 − 0.5)3

= 8 ,

41



from which,

π0 =18.


πi =18·

(0.5)i (i + 2)!2! i!

=(0.5)i+3 (i + 2)!

2! i!, i = 0, 1, 2, · · ·

(4) when β = 5, and n = 5/0.5 = 10,

1 +

∞∑i=1

λ0λ1 · · · λi−1

µ1µ2 · · · µi= 1 +

∞∑i=1

(0.5)i (i + 9)!9! i!

= 1 +(0.5) 10!

9! 1!+

(0.5)2 11!9! 2!

+(0.5)3 12!

9! 3!+ · · · (binomial series)

=

∞∑i=0

(0.5)i

(10 + i − 1

i

)

=1

(1 − 0.5)10

= 210 ,

from which,

π0 =

(12

)10

.


πi =

(12

)10

·(0.5)i (i + 9)!

9! i!=

(0.5)i+10 (i + 9)!9! i!

, i = 0, 1, 2, · · ·

Hence, we can deduce thatπ0 = (0.5)n .

However, for general parameter values, π0 is actually given by

π0 =

(1 −

λ

µ

)n

.

Since, in this problem, β/λ = n is an integer, then the m.g.f. corresponds to a negative binomialdistribution with parameter p = 1 − λ/µ, if and only if λ < µ and β > 0. The general formula forthe stationary probability distribution is then given by

42



πi =

(i + n − 1

i

)pn (1 − p)i , i = 0, 1, 2, · · · (40)

In the case n = 1, the negative binomial distribution is the same as the geometric distribution, aswe have seen with β = 0.5 and λ = 0.5. The graphs of the stationary probability distributions forfour different sets of the immigration rate β is shown in Fig. 4.

Figure 4: Stationary probability distributions for the simple birth, death, and immigration processwith λ = 0.5, µ = 1, and β = 0.5 (blue), 1.0 (magenta), 1.5 (green), and 5 (orange).

Using

m =β

µ − λand σ2 =

βµ

(µ − λ)2

for the mean and variance, respectively, then the mean values for the stationary probability distri-butions corresponding to each parameter set are m = 1, 2, 3, and 10, and the variance σ2 = 2, 4, 6,and 20.

The plots of population size over time with four sample paths of the birth and death process withmigration are shown in Fig. 5. The Python code to produce these results are given in the PythonImplementation section 3.4. The code are also available on the course website.

43



Figure 5: Four sample paths corresponding to the birth, death, and immigration process whenX(0) = 20, λ = 0.5, µ = 1, and β = 0.5, 1.0, 1.5, 5.

3 Python Implementation

All simulations were performed using Python v 3.6. All code presented here are available on thecourse website.

3.1 Yule Process Simulation

To simulate the Yule process, it is necessary to know the random variable for the time betweenbirths or interevent/sojourn time. It is shown that the random variable for the interevent/sojourntime is exponentially distributed; if the population is of size N, then the time T to the next event(or interevent/sojourn time) has a distribution equal to [1]

Prob{T ≥ h} = exp(−bNs) .

To simulate a value h ∈ T , a uniformly distributed random number Y from the uniform distribu-

44



tion U(0, 1) is selected in the range 0 ≤ Y ≤ 1 [1]. Then

Y = exp(−bNh) ,

and solving for h yields

h = −ln(Y)bN

.

Note that in the simple birth process, as N increases, h = − ln(Y)/bN decreases, or, the intereven-t/sojourn time decreases as the population size increases [1].

In Python, to generate a random sample Y from the uniform distribution, we use:

import numpy as np

Y = np.random.rand()

which is used to get h:

h = -np.log(Y)/(b*N)

or:

h = -np.log(np.random.rand())/(b*N)

What we are calculating here is the interevent/sojourn times as the population size increases oneby one, by using:

for j in range(numberofpaths):for i in range(N-1):

h = - np.log(np.random.rand())/(b*X[i])

s[j,i+1] = s[j,i] + h

and after which, the population is increased:

X[j,i+1] = X[j,i] + 1

The whole Python code for the Yule process can be written as follows:

import numpy as npimport matplotlib.pyplot as plt

np.random.seed(100)

def simplebirth(N, samplepaths , b, N0):s = np.zeros((samplepaths , N))

X = np.zeros((samplepaths , N))

X[:,0] = N0

45



for j in range(samplepaths):for i in range(N-1):

h = - np.log(np.random.rand())/(b*X[j,i])

s[j,i+1] = s[j,i] + h

X[j,i+1] = X[j,i] + 1

return [s, X]

def deterministic(s, N, b, N0):tmax = max(s[:, -1])

t = np.linspace(0, tmax, 10*N)

yd = N0*np.exp(b*t)

return[t, yd]

def solve_and_plot(N, paths, b, X0):fig, ax = plt.subplots()

[sojourn, population] = simplebirth(N, paths, b, X0)

[time, detsol] = deterministic(sojourn, N, b, X0)

## Sets axis ranges for plotting

xmax = max([max(sojourn[k,:]) for k in range(paths)])ymin = min([min(population[k,:]) for k in range(paths)])ymax = max([max(population[k,:]) for k in range(paths)])

## Generates plots

for r in range(walkers):plt.step(sojourn[r,:], population[r,:], where=’post’, label="Path %s

" % str(r+1))

plt.plot(time, detsol, ’k--’, label="Deterministic")

plt.axis([-0.2, xmax+0.2, ymin-2, ymax+2])

ax.set_xlabel(’Time’, fontsize=14)

ax.set_ylabel(’Population Size’, fontsize=14)

plt.xticks(fontsize=14)

plt.yticks(fontsize=14)

plt.tight_layout()

plt.legend(loc=2)

plt.show()

maxPop = 100

walkers = 3

birthrate = 0.5

initPop = 5

solve_and_plot(maxPop, walkers, birthrate , initPop)

46



The code for calculating probability distributions of the Yule process given in Eq. (27) is

import numpy as npimport matplotlib.pyplot as pltimport scipy.miscimport scipy.specialfrom matplotlib.lines import Line2Dfrom cycler import cycler

def probdist(N, b):t = np.arange(1, N)

k = np.arange(0, 51) # population vector

p = np.zeros((len(t), len(k)))

for i in range(len(t)):for j in range(len(k)):

p[i][j] = scipy.special.binom(N+k[j]-1, k[j])\

*np.exp(-b*N*t[i])\

*(1-np.exp(-b*t[i]))**k[j]

return [p, t, k]

def solve_and_plot(N, b):fig, ax = plt.subplots()

mycolors = [’cornflowerblue’, ’orchid’, ’olive’, ’gold’]

colors = ax.set_prop_cycle(cycler(’color’, mycolors))

marker_style = dict(color=colors, linestyle=’:’,

marker=’o’, markersize=6)

[p, t, k] = probdist(N, b)

for y, fill_style in enumerate(Line2D.fillStyles):for r in range(len(t)):

ax.plot(k, p[r][:], fillstyle=fill_style , **marker_style)

ax.set_xlabel(’Population Size $k$’, fontsize=14)

ax.set_ylabel(’Probability $P_k(t)$’, fontsize=14)

plt.legend([’$t=%s$’ % str(i+1) for i in range(len(t))], fontsize=14,loc=1)



plt.text(0.5, 0.9, ’$\lambda = %s$’ % b, fontsize=14,horizontalalignment=’center’,

verticalalignment=’center’,

transform=ax.transAxes)

plt.tight_layout()

plt.show()

N = 5

47



birth = 0.5

solve_and_plot(N, birth)

3.2 Linear Death Process Simulation


np.random.seed(100)

def simpledeath(N, samplepaths , d, N0):s = np.zeros((samplepaths , N))


X[:,0] = N0

for j in range(samplepaths):for i in range(N-1):

h = - np.log(np.random.rand())/(d*X[j,i])

s[j,i+1] = s[j,i] + h

X[j,i+1] = X[j,i] - 1

return [s, X]

def deterministic(s, N, d, N0):tmax = max(s[:,-1])

t = np.linspace(0, tmax, 10*N)

yd = N0*np.exp(-d*t)

return [t, yd]

def solve_and_plot(N, paths, d, X0):fig, ax = plt.subplots()

[sojourn, population] = simpledeath(N, paths, d, X0)

[time, detsol] = deterministic(sojourn, N, d, X0)


xmax = max([max(sojourn[k,:]) for k in range(paths)])ymax = max([max(population[k,:]) for k in range(paths)])

## Generates plots

for r in range(walkers):plt.step(sojourn[r,:], population[r,:], where=’pre’,

label="Path %s" % str(r+1))

plt.plot(time, detsol, ’k--’, label="Deterministic")

48



plt.axis([-0.1, xmax+0.2, 0, ymax+(ymax*0.01)])





plt.tight_layout()

plt.legend(loc=1)

plt.show()

maxPop = 20

walkers = 3

deathrate = 0.5

initPop = maxPop

solve_and_plot(maxPop, walkers, deathrate , initPop)

Note that the code for the Yule and linear death processes are almost identical, except for thefollowing:

• Initial population in the linear death process must be equal to its maximal population size,whereas in the Yule process the initial population must be smaller than the maximal popu-lation size,

• In the Yule process the population increases, X[j,i+1]=X[j,i]+1, and in the linear deathprocess the population decreases, X[j,i+1]=X[j,i]-1.

The code for probability distributions of linear death process using Eq. (32) is as follows:

import numpy as npimport matplotlib.pyplot as pltimport scipy.miscimport scipy.specialfrom matplotlib.lines import Line2Dfrom cycler import cycler

def probdist(N, mu):t = np.arange(1, 5)

k = np.arange(0, N+1)

p = np.zeros((len(t), len(k)))

for i in range(len(t)):for j in range(len(k)):

p[i][j] = scipy.special.binom(N, j)\

*np.exp(-j*mu*t[i])\

*(1-np.exp(-mu*t[i]))**(N-j)

return [p, t, k]

49



def solve_and_plot(N, mu):fig, ax = plt.subplots()

mycolors = [’cornflowerblue’, ’orchid’, ’olive’, ’gold’]

colors = ax.set_prop_cycle(cycler(’color’, mycolors))

marker_style = dict(color=colors, linestyle=’:’,

marker=’o’, markersize=6)

[p, t, k] = probdist(N, mu)

for y, fill_style in enumerate(Line2D.fillStyles):for r in range(len(t)):

ax.plot(k, p[r][:], fillstyle=fill_style , **marker_style)

ax.set_xlabel(’Population Size $k$’, fontsize=14)

ax.set_ylabel(’Probability $P_k(t)$’, fontsize=14)

plt.legend([’$t=%s$’ % str(i+1) for i in range(len(t))],fontsize=14, loc=1)



plt.text(0.5, 0.9, ’$\mu = %s$’ % mu, fontsize=14,horizontalalignment=’center’,



plt.tight_layout()

plt.show()

N = 20

death = 0.5

solve_and_plot(N, death)

3.3 Birth and Death Process Simulation

For birth and death processes, we first look at the transition probabilities


=

1 − (µi + λi)∆t + o(∆t) j = 0

λi∆t + o(∆t) j = 1

µi∆t + o(∆t) j = −1

o(∆t) j , − 1, 0, 1 .

(41)

50



According to Eq. (41), during a small time duration of length ∆t, a transition occurs from state ito state i+1 with probability λi∆t+o(∆t) and from state i to state i−1 with probability µi∆t+o(∆t).Given that a transition occurs at time t, the probability of:

• moving to state i + 1 isλi

λi + µi

• moving to state i − 1 isµi

λi + µi

Recall that the random variable for the interevent/sojourn time is continuous and nonnegative,that is Ti = Wi+1 − Wi ≥ 0, where Wi is the time of the ith jump. In the linear birth and deathprocess, if the events consist of births with rate λi = iλ and deaths with rate µi = iµ, the intereventtime Ti is given by

Ti = −ln(U1)

(λ + µ)i,

where U1 is a generated random number.

Since there are two events, to determine whether there is a birth or a death, the unit interval [0, 1]is divided into two subintervals:

• one subinterval has probability

λi

λi + µi=

iλiλ + iµ

=λ

λ + µ,

in which a birth occurs, and

• the other has probability probability

µi

λi + µi=

iµiλ + iµ

=µ

λ + µ

in which a death occurs.

We generate another random number, say U2, and if U2 < λ/(λ + µ), then this random numberlies in the first subinterval and a birth occurs. Otherwise, if U2 ≥ λ/(λ + µ), the random numberlies in the second subinterval and a death occurs.

The deterministic analogue of this simple birth and death process is the differential equationdn/dt = (λ − µ)n, with initial condition n(0) = N, and the solution is given by

n(t) = N e(λ−µ)t .

51




np.random.seed(100)

def simplebirthdeath(N, samplepaths , b, d):X0 = 50 # initial population size

s = np.zeros((samplepaths , N))


X[:,0] = X0

s[:,0] = 0.001

for j in range(samplepaths):i = 0

while X[j,i] > 0 and i < (N-1):U1 = np.random.rand()

U2 = np.random.rand()

h = - np.log(U1)/((b+d)*X[j,i])

s[j,i+1] = s[j,i] + h

if U2 < b/(b+d):X[j,i+1] = X[j,i] + 1 # birth occurs

else:X[j,i+1] = X[j,i] - 1 # death occurs

i += 1

return [s, X]

def solve_and_plot(N, paths, b, d):fig, ax = plt.subplots()

[sojourn, population] = simplebirthdeath(N, paths, b, d)


xmax = max([max(sojourn[k,:]) for k in range(paths)])ymax = max([max(population[k,:]) for k in range(paths)])

sojourn[sojourn==0] = np.nan

## Generates plots

for r in range(paths):plt.step(sojourn[r,:], population[r,:], where=’pre’, label="Path %s"

% str(r+1))

plt.axis([-0.1, xmax+0.2, 0, ymax+2])




52




plt.tight_layout()

plt.legend(loc=1)

plt.show()

maxPop = 2000

walkers = 3

birth = 1.0

death = 1.0

solve_and_plot(maxPop, walkers, birth, death)

3.4 Birth and Death Process with Migration Simulation

A birth and death process with (im)migration is called a linear growth process if the increase ofa population is determined by λi = iλ + β and the decrease of population by µi = iµ, with λ > 0,β > 0, and µ > 0. Here β represents the rate of immigration into the population and λ representsthe individual birth rate.

For this process, there are three events and to determine whether population increases or de-creases, the unit interval [0, 1] is divided into three subintervals:

• the first subinterval has probability

iλiλ + β + iµ

in which a birth occurs,

• the first subinterval has probability probability

iµiλ + β + iµ

in which a death occurs, and

• the third subinterval has probability

β

iλ + β + iµ

in which an immigration occurs.

The endpoints of the unit interval are:

0,iλ

i(λ + µ) + β,

i(λ + µ)i(λ + µ) + β

, 1.

53



The unit interval [0, 1] having three subintervals with each probability and four end points isdepicted in the following figure:

If a generated random number, say U2, is less than iλ/(i(λ+µ) +β), then the random number liesin the first subinterval and a birth occurs. If U2 ≥ iλ/(i(λ+µ)+β) and U2 < i(λ+µ)/(i(λ+µ)+β),the random number lies in the second subinterval and a death occurs. Otherwise, if U2 ≥ i(λ +

µ)/(i(λ + µ) + β), the random number lies in the third subinterval and an immigration occurs.

In general, if there are k events with rates αk(n), k = 1, 2, 3, · · · , where n is the current state, theunit interval must be divided into k subintervals (and k + 1 endpoints). The probabilities of theseevents are

αk(n)α(n)

, α(n) =∑

k

αk(n) , k = 1, 2, 3, · · ·

The python code for the simple birth and death process with immigration is as follows


np.random.seed(10)

def birthdeathmigration(b, d, c, paths):X0 = 20 # initial population size

N = 200 # maximal population size

# Pre-allocation and initialization

s = np.zeros((paths, N))

X = np.zeros((paths, N))

X[:,0] = X0

s[:,0] = 0.00

for j in range(paths):i = 0

while X[j,i] >= 0 and i < (N-1):U1 = np.random.rand()

U2 = np.random.rand()

h = - np.log(U1)/((b+d)*X[j,i]+c)

s[j,i+1] = s[j,i] + h

54



if U2 < b*X[j,i]/((b+d)*X[j,i]+c):X[j,i+1] = X[j,i] + 1 # a birth occurs

elif U2 >= b*X[j,i]/((b+d)*X[j,i]+c)\and U2 < (b+d)*X[j,i]/((b+d)*X[j,i]+c):

X[j,i+1] = X[j,i] - 1 # a death occurs

elif U2 >= (b+d)*X[j,i]/((b+d)*X[j,i]+c):X[j,i+1] = X[j,i] + 1 # immigration occurs

i += 1

return [X, s]

def solve_and_plot(b, d, c, paths):fig, ax = plt.subplots()

[population , sojourn] = birthdeathmigration(b, d, c, paths)


xmax = max([max(sojourn[k,:]) for k in range(paths)])

## Generates plots

for r in range(paths):plt.step(sojourn[r,:], population[r,:],

where=’pre’, label="Path %s" % str(r+1))

plt.axis([-0.2, xmax+0.2, -2, 32]) #ymax+2])



plt.text(0.5, 0.9, ’$\\beta = %s$’ % c, fontsize=14,horizontalalignment=’center’,





plt.tight_layout()

plt.legend(loc=1)

plt.grid(True)

plt.show()

## Uncomment to save the figure

#fig.savefig(’Figures/bd_mig_1p0.png’, dpi=600)

b = 0.5 # birth rate

d = 1.0 # death rate

c = 1.0 # immigration rate

walkers = 4

solve_and_plot(b, d, c, walkers)

55



References

[1] Linda J. S. Allen. An Introduction to Stochastic Processes with Applications to Biology2nd Edition. CRC Press, 2010.

[2] Paul G. Hoel, Sidney C. Port, Charles J. Stone. Introduction to Stochastic Processes.Houghton Mifflin Company, 1972.

[3] Samuel Karlin, Howard M. Taylor. A First Course in Stochastic Processes. AcademicPress, 1975.

[4] Samuel Karlin, Howard M. Taylor. A Second Course in Stochastic Processes. AcademicPress, 1981.

[5] Gregory F. Lawler. Introduction to Stochastic Processes 2nd Edition. Chapman & Hall,2006.

[6] Mario Lefebvre. Applied Stochastic Processes. Springer, 2007.

[7] Sheldon M. Ross. Introduction to Probability Models 9th Edition. Elsevier, 2007.

[8] Sheldon M. Ross. Stochastic Processes 2nd Edition. John Wiley & Sons Inc., 1996.

[9] Howard M. Taylor, Samuel Karlin. An Introduction to Stochastic Modeling 3rd Edition.Academic Press, 1998.

[10] Henry C. Tuckwell. Elementary Applications of Probability Theory 2nd Edition. Springer-Science+Business Media, BV, 1995.

[11] Law of Total Probability,https://en.wikipedia.org/wiki/Law of total probability

[12] Point Process,https://en.wikipedia.org/wiki/Point process

[13] Counting Processeshttps://www.probabilitycourse.com/chapter11/11 1 1 counting processes.php

56


Documents

Lecture 4b: Continuous-Time Birth and Death Processespeople.bu.edu/andasari/courses/stochasticmodeling/... · the population is 0, there are no deaths and 0 = 0 [5,7]. 2.1 General