Renewal theory and its applications - TU/eresing/isp/ISP2.pdf · Renewal theory and its applications ... equals the number of events in (s;t]. ... Theorem (Elementary Renewal Theorem)

Renewal theory and its applications

Renewal theory and itsapplications

Stella Kapodistria and Jacques Resing

September 11th, 2012

ISP


Definition of a Renewal process


If we substitute the Exponentially distributed inter-arrival times of thePoisson process by any arbitrary sequence of iid r.v. {X1,X2, . . .} we cangeneralize the definition of the counting process.

DefinitionIf the sequence of nonnegative random variables {X1,X2, . . .} isindependent and identically distributed, then the counting process{N(t), t ≥ 0} is said to be a renewal process.

For any sequence of iid r.v. we can define a counting process as

N(t) = max{n : Sn ≤ t} withn∑

j=1

Xj = Sn

The definition implies:

(i) N(t) ≥ 0

(ii) N(t) is integer valued

(iii) If s < t, then N(s) ≤ N(t)

(iv) For s < t, N(t)− N(s) equals the number of events in (s, t].


Definition of a Renewal process

Poisson process

DefinitionThe counting process {N(t), t ≥ 0} is called a Poisson process with rateλ, if {X1,X2, . . .} are iid having the exponential distribution at rate λ.

In this case we know that:

(i) N(t) ∼ Poisson(λt) and m(t) = E [N(t)] = λt

(ii)∑n

j=1 Xj ∼ Erlang(λ, n) and E [Sn] = n/λ

(iii) By the Strong Law of Large Numbers (SLLN) it follows that,

Sn

n→ 1

λ, n→∞ (w.p.1)


Distribution of N(t)


The distribution of N(t) can be obtained, at least in theory, by notingthat

N(t) ≥ n⇐⇒ Sn ≤ t

Then,

P[N(t) = n] = P[N(t) ≥ n]− P[N(t) ≥ n + 1]

= P[Sn ≤ t]− P[Sn+1 ≤ t]

Now since the random variables Xi , i ≥ 1, are iid having a distribution F ,it follows that Sn =

∑nj=1 Xj is distributed as Fn, the n-fold convolution

of F with itself (Section 2.5). Therefore, we obtain

P[N(t) = n] = Fn(t)− Fn+1(t)



The mean-value function


The mean-value function can be obtained by noting that

N(t) ≥ n⇐⇒ Sn ≤ t

Then,

m(t) = E [N(t)] =∞∑n=1

P[N(t) ≥ n]

=∞∑n=1

P[Sn ≤ t]

=∞∑n=1

Fn(t)

The function m(t) is known as the mean-value or the renewal function.

Example

Suppose m(t) = 2t. What is the distribution P[N(10) = n] =?





There is one-to-one correspondence between the renewal process and itsmean-value function!

We define m̃(s) to be the Laplace-Stieltjes transform of m(t)

m̃(s) =

∫ ∞0

e−stm′(t)dt

Then we can prove that

m̃(s) =ψ(s)

1− ψ(s)

with ψ(s) = E [e−sX ] we have denoted the Laplace-Stieltjes transform ofthe inter-arrival times {X1,X2, . . .}.





There is one-to-one correspondence between the renewal process and itsmean-value function!

We want to determine m(t) for t ≥ 1. We will prove a basic renewalequation as follows

m(t) = E [N(t)] =

∫ ∞0

E [N(t)|X1 = x ]f (x)dx

=

∫ t

0

E [N(t)|X1 = x ]f (x)dx +

∫ ∞t

E [N(t)|X1 = x ]f (x)dx

=

∫ t

0

[1 + m(t − x)]f (x)dx +

∫ ∞t

0f (x)dx

= F (t) +

∫ t

0

m(t − x)f (x)dx

This last equation is called the renewal equation and can sometimes besolved to obtain the mean-value function.




Proposition 7.2 E [SN(t)+1] = (m(t) + 1)µ

Proposition (7.2)

E [SN(t)+1] = (m(t) + 1)µ

Remark (Wald’s Equation)

For any sequence {X1,X2, . . .} of iid r.v. with mean E [X ] = µ and a r.v.N independent from the sequence {X1,X2, . . .} we can easily prove that

E [N∑i=1

Xi ] = E [N]E [X ] = E [N]µ





Proposition (7.2)

E [SN(t)+1] = (m(t) + 1)µ

Proof.We define E [SN(t)+1] = g(t), then

g(t) =

∫ ∞0

E [SN(t)+1|X1 = x ]f (x)dx

=

∫ t

0

E [SN(t)+1)|X1 = x ]f (x)dx +

∫ ∞t

E [SN(t)+1|X1 = x ]f (x)dx

=

∫ t

0

[x + g(t − x)]f (x)dx +

∫ ∞t

xf (x)dx

= µ+

∫ t

0

g(t − x)f (x)dx





Proposition (7.2)

E [SN(t)+1] = (m(t) + 1)µ


g(t) = µ+

∫ t

0

g(t − x)f (x)dx

If we substitute g(t) = (m(t) + 1)µ yields

m(t) = F (t) +

∫ t

0

m(t − x)f (x)dx

which completes the proof.





Proposition (7.2)

E [SN(t)+1] = (m(t) + 1)µ


g(t) = µ︸︷︷︸k(t)

+

∫ t

0

g(t − x)f (x)dx

For any known function k(t) the renewal type equation has a uniquesolution:

g(t)= k(t) +

∫ t

0

k(t − x)m′(x)dx

(m(t) = F (t) +

∫ t

0

m(t − x)F (x)dx)

Then by setting k(t) = µ immediately yields the result.


Limit Theorems

Limit Theorems

Let {X1,X2, . . .} be a sequence of iid r.v. and we define the renewalprocess {N(t), t ≥ 0} as

N(t) = max{n : Sn ≤ t} withn∑

j=1

Xj = Sn

Let E [Xj ] = µ.

By the SLLN it follows that,

Sn

n→ µ, n→∞ (w.p.1)

Hence, Sn →∞ as n→∞. Thus, Sn ≤ t for at most a finite number ofvalues of n, and hence by definition N(t) must be finite. However,though N(t) <∞ for each t it is true that, with probability 1,

N(∞) = limt→∞

N(t) =∞


Limit Theorems

Limit Theorems

Proposition ( 7.1)

With probability 1,N(t)

t→ 1

µas t →∞

Proof.First of all recall that E [SN(t)] = E [N(t)]E [X ], hence by the SLLN

SN(t)/N(t)→ E [X ] = µ t →∞ (w.p.1)

Secondly,

SN(t) ≤ t < SN(t)+1

SN(t)

N(t)≤ t

N(t)<

SN(t)+1

N(t)∑N(t)j=1 Xj

N(t)≤ t

N(t)<

SN(t)+1

N(t) + 1

N(t) + 1

N(t)=

∑N(t)+1j=1 Xj

N(t) + 1

N(t) + 1

N(t)


Limit Theorems

Limit Theorems

Proposition ( 7.1)

With probability 1,N(t)

t→ 1

µas t →∞

Proof.First of all recall that E [SN(t)] = E [N(t)]E [X ], hence by the SLLN

SN(t)/N(t)→ E [X ] = µ t →∞ (w.p.1)

Secondly,

SN(t) ≤ t < SN(t)+1

SN(t)

N(t)≤ t

N(t)<

SN(t)+1

N(t)

��>

µ∑N(t)j=1 Xj

N(t)≤ t

N(t)<

SN(t)+1

N(t) + 1

N(t) + 1

N(t)=��

��*µ∑N(t)+1

j=1 Xj

N(t) + 1 ��*1

N(t) + 1

N(t)


Limit Theorems

Limit Theorems

Theorem (Elementary Renewal Theorem)

m(t)

t→ 1

µas t →∞

Theorem (Central Limit Theorem for Renewal Process)

limt→∞

P

[N(t)− t/µ√

tσ2/µ3< x

]=

1√2π

∫ x

−∞e−x

2/2dx

with µ = E [X ] and σ2 = Var [X ].

Remark

limt→∞

Var [N(t)]

t=σ2

µ3


Limit Theorems

Example 7.7

Example 7.7

Suppose that potential customers arrive at a single-server bank inaccordance with a Poisson process having rate λ. Furthermore, supposethat the potential customer will enter the bank only if the server is freewhen he arrives; if upon arrival the customer sees the bank telleroccupied he will immediately leave. If we assume that the amount oftime spent in the bank by an entering customer is a random variablehaving distribution G , then(a) what is the rate at which customers enter the bank?(b) what proportion of potential customers actually enter the bank?

In answering these questions, suppose that at time 0 a customer has justentered the bank.


Limit Theorems

Example 7.9

Example 7.9

Consider the renewal process whose inter-arrival distribution is theconvolution of two exponentials; that is, F = F1 ∗ F2, whereFi (t) = 1− e−µi t , i = 1, 2. Imagine that each renewal corresponds to anew machine being put in use, and suppose that each machine has twocomponentsinitially component 1 is employed and this lasts anexponential time with rate µ1, and then component 2, which functionsfor an exponential time with rate µ2, is employed. When component 2fails, a new machine is put in use (that is, a renewal occurs). Nowconsider the process {X (t), t ≥ 0} where X (t) is i if a type i componentis in use at time t. Calculate(a) the probability that the machine in use at time t is using its firstcomponent.(b) the expected excess time E [Y (t)] := E [SN(t)+1 − t].(c) the mean-value function.

In answering these questions, suppose that at time 0 a component 1 hasjust been employed.


Limit Theorems

Example 7.10

Example 7.10

Two machines continually process an unending number of jobs. The timethat it takes to process a job on machine 1 is a Gamma random variablewith parameters n = 4, λ = 2, whereas the time that it takes to processa job on machine 2 is Uniformly distributed between 0 and 4.Approximate the probability that together the two machines can processat least 90 jobs by time t = 100.


Renewal Reward Processes


Consider a renewal process {N(t), t ≥ 0} having inter-arrival times{X1,X2, . . .} and suppose that each time a renewal occurs we receive areward. We denote by Rn, the reward earned at the time of the n-threnewal. We assume that the Rn, n ≥ 1, are iid r.v. If we let

R(t) =

N(t)∑n=1

Rn

then R(t) represents the total reward earned by time t. Let

E [R] = E [Rn], E [X ] = E [Xn]

Proposition (7.3)

If E [R] <∞ and E [X ] <∞, then

a) w.p.1 limt→∞

R(t)t = E [R]

E [X ] b) limt→∞

E [R(t)]t = E [R]

E [X ]



Example 7.11

Example 7.11

Suppose that potential customers arrive at a single-server bank inaccordance with a Poisson process having rate λ. However, suppose thatthe potential customer will enter the bank only if the server is free whenhe arrives. That is, if there is already a customer in the bank, then ourarriver, rather than entering the bank, will go home. If we assume thatthe amount of time spent in the bank by an entering customer is arandom variable having distribution G , and

that each customer that enters makes a deposit and that the amountsthat the successive customers deposit in the bank are iid r.v. having acommon distribution H, then the rate at which deposits accumulate —that is, limt→∞(total deposits by the time t)/t — is given by

E [deposits during a cycle]

E [time of cycle]=

µH

µG + 1/λ

where µG + 1/λ is the mean time of a cycle, and µH is the mean of thedistribution H.



Example 7.16

Example 7.16 (The Average Age of a Renewal Process)

Consider a renewal process having inter-arrival distribution F and defineA(t) := t − SN(t) at time t. We are interested in

lims→∞

∫ s

0A(t)dt

s= average value of age

Assume that∫ s

0A(t)dt represents our total earnings by time s:

lims→∞

∫ s

0A(t)dt

s→ E [reward during a renewal cycle]

E [time of a renewal cycle]

Now since the age of the renewal process a time t into a renewal cycle isjust t, we have

reward during a renewal cycle =

∫ X

0

tdt

where X ∼ F is the time of the renewal cycle. Hence, we have that

average value of age = lims→∞

∫ s

0A(t)dt

s=

E [X 2]/2

E [X ]


Regenerative Processes


Consider a stochastic process {X (t), t ≥ 0} with state space{0, 1, 2, . . .}, having the property that there exist time points at whichthe process (probabilistically) restarts itself. That is, suppose that withprobability one, there exists a time T1, such that the continuation of theprocess beyond T1 is a probabilistic replica of the whole process startingat 0. Note that this property implies the existence of further timesT2,T3, . . ., having the same property as T1. Such a stochastic process isknown as a regenerative process.

From the preceding, it follows that T1,T2, . . ., constitute the arrivaltimes of a renewal process, and we shall say that a cycle is completedevery time a renewal occurs.

Example

(1) A renewal process is regenerative, and T1 represents the time of thefirst renewal.(2) A recurrent Markov chain is regenerative, and T1 represents the timeof the first transition into the initial state.




We are interested in determining the long-run proportion of time that aregenerative process spends in state j .

To obtain this quantity, let us imagine that we earn a reward at a rate 1per unit time when the process is in state j and at rate 0 otherwise. Thatis, if I (s) represents the rate at which we earn at time s, then

I (s) =

{1, if X (s) = j

0, if X (s) 6= j

and

total reward earned by t =

∫ t

0

I (s)ds

Proposition (7.4)

For a regenerative process, the long-run proportion of time in state j

=E [amount of time in j during a cycle]

E [time of a cycle]



Example 7.18

Example 7.18

Consider a positive recurrent continuous time Markov chain that isinitially in state i . By the Markovian property, each time the processreenters state i it starts over again. Thus returns to state i are renewalsand constitute the beginnings of new cycles. By Proposition 7.4, itfollows that

the long-run proportion of time in state j

=E [amount of time in j during an i–i cycle]

E [Ti,i ]

where E [Ti,i ] represents the mean time to return to state i . If we take jto equal i , then we obtain

proportion of time in state i =1/vi

E [Ti,i ]



Example 7.19

Example 7.19 (A Queueing System with Renewal Arrivals)

Consider a waiting time system in which customers arrive in accordancewith an arbitrary renewal process and are served one at a time by a singleserver having an arbitrary service distribution. If we suppose that at time0 the initial customer has just arrived, then {X (t), t ≥ 0} is aregenerative process, where X (t) denotes the number of customers in thesystem at time t. The process regenerates each time a customer arrivesand finds the server free.



Alternating Renewal Processes

Alternating Renewal Processes

Consider a system that can be in one of two states: on or off. Initially itis on, and it remains on for a time Z1; it then goes off and remains off fora time Y1. It then goes on for a time Z2; then off for a time Y2; then on,and so on.

We suppose that the random vectors (Zn,Yn), n ≥ 1 are iid; but we allowZn and Yn to be dependent. In other words, each time the process goeson, everything starts over again, but when it then goes off, we allow thelength of the off time to depend on the previous on time.

We are concerned with Pon, the long-run proportion of time that thesystem is on

Pon =E [Z ]

E [Y ] + E [Z ]=

E [on]

E [off] + E [on]



Example 7.23

Example 7.23 (The Age of a Renewal Process)

Suppose we are interested in determining the proportion of time that theage of a renewal process is less than some constant c . To do so, let acycle correspond to a renewal, and say that the system is “on at time t ifthe age at t is less than or equal to c , and say it is “off” if the age at t isgreater than c . In other words, the system is on the first c time units ofa renewal interval, and off the remaining time. Hence, letting X denote arenewal interval, we have

proportion of time age is less than c =E [min(X , c)]

E [X ]

=

∫∞0

P[min{X , c} > x ]dx

E [X ]

=

∫ c

0P[X > x ]dx

E [X ]

=

∫ c

0(1− F (x))dx

E [X ]



Example 7.23

Summary Renewal theory

1 Definition of a Renewal process

2 Distribution of N(t)The mean-value function

3 Limit TheoremsExample 7.7Example 7.9Example 7.10

4 Renewal Reward ProcessesExample 7.11Example 7.16

5 Regenerative ProcessesExample 7.18Example 7.19Alternating Renewal ProcessesExample 7.23



Example 7.23

Exercises

Introduction to Probability ModelsHarcourt/Academic Press, San Diego, 9th ed., 2007Sheldon M. Ross

Chapter 7

Sections 7.1, 7.2, 7.3, 7.4, 7.5

Exercises: 2, 4, 5, 10, 11, 12, 15, 19, 26, 32, 37, 44

Documents

Renewal theory and its applications - TU/eresing/isp/ISP2.pdf · Renewal theory and its applications ... equals the number of events in (s;t]. ... Theorem (Elementary Renewal Theorem)