1

Queueing Theory

Frank Y. S. LinInformation Management Dept.

National Taiwan Universityyslin@im.ntu.edu.tw

2

ReferencesLeonard Kleinrock, “Queueing Systems Volume I: Theory”, New York: Wiley, 1975-1976.

D. Gross and C. M. Harris, “Fundamentals of Queueing Theory”, New York: Wiley, 1998.

3

AgendaIntroductionStochastic ProcessGeneral ConceptsM/M/1 ModelM/M/1/K ModelDiscouraged ArrivalsM/M/∞ and M/M/m ModelsM/M/m/m Model

4

Introduction

5

Queueing SystemA queueing system can be described as customers arriving for service, waiting for service if it is not immediate, and if having waited for service, leaving the system after being served.

6

Why Queueing TheoryPerformance Measurement

Average waiting time of customer / distribution of waiting time.Average number of customers in the system / distribution of queue length / current work backlog.Measurement of the idle time of server / length of an idle period.Measurement of the busy time of server / length of a busy period.System utilization.

7

Why Queueing Theory (cont’d)Delay AnalysisNetwork Delay =

Queueing Delay + Propagation Delay (depends on the distance)+ Node Delay Processing Delay

(independent of packet length, e.g. header CRC check)

Adapter Delay (constant)

8

Characteristics of Queueing ProcessArrival Pattern of Customers

Probability distributionPatient / impatient (balked) arrivalStationary / nonstationary

Service PatternsProbability distributionState dependent / independent serviceStationary / nonstationary

9

Characteristics of Queueing Process (cont’d)

Queueing DisciplinesFirst come, first served (FCFS)Last come, first served (LCFS)Random selection for service (RSS)Priority queuePreemptive / nonpreemptive

System CapacityFinite / infinite waiting room.

10

Characteristics of Queueing Process (cont’d)

Number of Service ChannelsSingle channel / multiple channelsSingle queue / multiple queues

Stages of ServiceSingle stage (e.g. hair-styling salon)Multiple stages (e.g. manufacturing process)Process recycling or feedback

11

NotationA queueing process is described by A/B/X/Y/Z

12

Notation (cont’d)For example, M/D/2/∞/FCFS indicates a queueing process with exponential inter-arrival time, deterministic service times, two parallel servers, infinite capacity, and first-come, first-served queueing discipline.Y and Z can be omitted if Y = ∞ and Z = FCFS.

13

Stochastic Process

14

Stochastic ProcessStochastic process: any collection of random variables Χ(t), t T, on a common probability space where t is a subset of time.

Continuous / discrete time stochastic processExample: Χ(t) denotes the temperature in the class on t = 7:00, 8:00, 9:00, 10:00, … (discrete time)

We can regard a stochastic process as a family of random variables which are “indexed” by time.For a random process X(t), the PDF is denoted by FX(x;t) = P[X(t) <= x]

∈

15

Some Classifications of Stochastic Process

Stationary Processes: independent of timeFX (x; t + τ) = FX (x; t)

Independent Processes: independent variablesFX (x; t) = FX1,…, Xn(x1,…, xn ; t1,…,tn)

= FX1(x1; t1) …FXn(xn; tn)

Markov Processes: the probability of the next state depends only upon the current state and not upon any previous states.

P[X(tn+1) = xn+1 | X(tn) = xn, …., X(t1) = x1]= P[X(tn+1) = xn+1 | X(tn) = xn]

16

Some Classifications of Stochastic Process (cont’d)

Birth-death Processes: state transitions take place between neighboring states only.

Random Walks: the next position the process occupies is equal to the previous position plus a random variable whose value is drawn independently from an arbitrary distribution.

17

General Concepts

18

Continuous-time Memoryless Property

If X ~Exp(λ), for any a,b > 0, PP[[X > a + b | X > aX > a + b | X > a] ] = P= P[[X > bX > b]]

Proof:P[X > a + b | X > a]( ) ( )

( ) ( ) ( )[ ]P X a b X aX a b X a

P X a> + >

= > + ⊂ >>I

( )( )

( )( )

( )1(

1)

a bx b

ax

P X a b F a b e eP X a F

Pe

Xa

bλ

λλ

− +−

−

> + − += = = = = >

> −

19

Global Balance EquationDefine Pi = P[system is in state i] Pij = P[get into state j right after leaving state i]

rate out of state j = rate into state j

0 0( ) ( )

j ij i iji ii j i j

P P P P∞ ∞

= =≠ ≠

⋅ = ⋅∑ ∑

20

General Balance EquationDefine S = a subset of the state space

0 0 0 0( ) ( ) ( ) ( )

j ij i ijj i i jj s i s i s j s

P P P P∞ ∞ ∞ ∞

= = = =∈ ∉ ∉ ∈

⋅ = ⋅∑ ∑ ∑ ∑

S

j

rate in = rate out

21

General Equilibrium SolutionNotation:

Pk = the probability that the system contains kcustomers (in state k)

λk= the arrival rate of customers when the system is in state k.μk= the service rate when the system is in state k.

01k

kP

∞

=

=∑

22

General Equilibrium Solution (cont’d)Consider state ≤ k:

k-1 k k+1

λk-1

μk μk+1

λk

1 1k k k kP Pλ μ+ +⋅ = ⋅

rate in rate out=

11

kk k

k

P Pλμ+

+

⇒ =

11

kk k

k

P Pλμ−

−⇒ =

.

.

.1

1 2 00 0

01 1 1

kk k i

kik k i

P P Pλ λ λ λμ μ μ μ

−− −

=− +

⋅ ⋅ ⋅ ⋅⇒ = = ⋅

⋅ ⋅⋅ ⋅ ∏#

23

General Equilibrium Solution (cont’d)

01k

kP

∞

=

=∑1

00 0 1

1k

i

k i i

Pλμ

−∞

= = +

⇒ ⋅ =∑∏

0 1

0 0 1

1

1k

i

k i i

Pλμ

−∞

= = +

∴ =+∑∏ 0

,k kk

NP Tλ λλ

∞

=

= =∑

1waiting time w Tμ

= −

24

Little’s Result= average number of customers in the system

T = system time (service time + queueing time)λ= arrival rate

N

N Tλ=

Black box

Nλ

System time T

…

Queueing time

Service time

25

M/M/1 Model

Single Server, Single Queue(The Classical Queueing System)

26

M/M/1 QueueSingle server, single queue, infinite population:

Interarrival time distribution:

Service time distribution

Stability condition λλ < < μμ

( ) tp t e λλ λ −=

00

0 0( ) 1

t ttp t t e dt e μμμ μ −−< = = −∫

kλ λ=kμ μ=

27

M/M/1 Queue (cont’d)System utilization

Define state Sn = n customers in the system(n-1 in the queue and 1 in service)S0 = empty system

= P[system is busy], 1- P[system is idel]λρ ρμ

= =

rate out

rate in

S

28

M/M/1 Queue (cont’d)Define pn = P[n customers in the system]

(rate in = rate out)

Since

1n np pλ μ +× = ×

1n n np p pλ ρμ+ = × = ×

11 0

nnp pρ ++ = ×

01i

ip

∞

=

=∑ 00

1n

ip ρ

∞

=

=∑ 00

1n

ip ρ

∞

=

=∑

0 1 , (1 )nnp pρ ρ ρ= − = × −

#

29

M/M/1 Queue (cont’d)Average number of customers in the system

(1 ) kN k ρ ρ= ⋅ −∑ (1 ) kkρ ρ= − ⋅∑(1 ) /kd dρ ρ ρ ρ= − ⋅ ⋅∑(1 ) kd

dρ ρ ρ

ρ= − ⋅ ⋅ ∑

1(1 )1

dd

ρ ρρ ρ⎛ ⎞

= − ⋅ ⋅ ⎜ ⎟−⎝ ⎠

1ρρ

=−

1N ρ

ρ∴ =

−#

30

M/M/1 Queue (cont’d)Average system time

P[≧ k customers in the system]

(Little’s Result)NTλ

=

(1 ) (1 )1

ki k

i k

ρρ ρ ρ ρρ

∞

=

= − = − =−∑

#

1/ 111

ρμρ

λ ρ μ λ−= = =

− −

31

M/M/1/K Model

Single Server, Finite Storage

32

M/M/1/K ModelThe system can hold at most a total of Kcustomers (including the customer in service)λk = λ if k < K

0 if k ≥ Kμk = μ

33

M/M/1/K Model (cont’d)1

0 00

kk

ki

P P P k Kλ λμ μ

−

=

⎛ ⎞= = ≤⎜ ⎟

⎝ ⎠∏

0kP k K= >

1

0 11

1 /1 ( / ) 01 ( / )

Kk

Kk

P k Kλ μλ μλ μ

−

+=

−⎡ ⎤⇒ = + = ≤ ≤⎢ ⎥ −⎣ ⎦

∑

0 otherwise

34

Discouraged Arrivals

35

Discouraged ArrivalsArrivals tend to get discouraged when more and more people are present in the system.

1k kαλ =+

kμ μ=

36

Discouraged Arrivals (cont’d)

( )1

0 00

/( 1) 1/!

kk

ki

iP P Pk

α α μμ

−

=

+= ⋅ = ⋅ ⋅∏

( )0

1

111 /!

k

k

P e

k

αμ

α μ

−

∞

=

= =+ ⋅∑( )/

!

k

kP ek

αμα μ −

⇒ = ⋅ N αμ

∴ =

37

Discouraged Arrivals (cont’d)( / )

0 0

( / )1 !k k

k kP e

k kα μα α μλ λ

∞ ∞−

= =

= = ⋅ ⋅+∑ ∑

( / )01 ( , 1 )e Pα μμ λ μρ ρ−⎡ ⎤= − = = −⎣ ⎦ Q

/

/(1 )

NTe α μ

α μλ μ −= =

−

38

M/M/∞ and M/M/m

M/M/∞ - Infinite Servers, Single Queue(Responsive Servers)

M/M/m - Multiple Servers, Single Queue(The m-Server Case)

39

M/M/∞ Queue

kλ λ=

k kμ μ=

There is always a new server available for each arriving customer.

40

M/M/∞ Queue (cont’d)1

/0

0

( / )( 1) !

kk

ki

P P ei k

λ μλ λ μμ

−−

=

= =+∏

N λμ

⇒ =

1Tμ

⇒ = (Little’s Result)

41

M/M/m QueueThe M/M/m queue

An M/M/m queue is shorthand for a single queue served by multiple servers.Suppose there are m servers waiting for a single line. For each server, the waiting time for a queue is a system with service rateμ and arrival rate λ/m.The M/M/1 analysis has been done, at risk conclusion:delay =

throughput

1/ nμ λ−

/ nn

λ λρμ μ

= =

42

M/M/m Queue (cont’d)

λk = λμk = kμ if k ≤ m

mμ if k > m

For k ≤ m

For k > m

0 01( )

2 !k

kp p pk k

λ λ λ λμ μ μ μ

= ⋅⋅⋅ =

0 01 1( ) ( )

2 !k k n

kp p pn n n n

λ λ λ λ λμ μ μ μ μ

−= ⋅⋅⋅ ⋅ ⋅ ⋅ =

43

M/M/n Queue (cont’d)

∴

P[queueing] =

Total system time =

01i

ip

∞

=

=∑

0 1

0

1( ) ( ) 1

! ! (1 )

k nn

ik

p wherenp np npk n

λρμ

ρ

−

=

= =+

−∑

kk m

p∞

=∑

02

1 (/ )( 1)!( )

n

pn n

λ μ μμ μ λ+ ×

− −

44

Comparisons (cont’d)M/M/1 v.s M/M/4If we have 4 M/M/1 systems: 4 parallel communication links that can each handle 50 pps (μ), arrival rate λ = 25 pps per queue.

average delay = 40 ms.Whereas for an M/M/4 system,

average delay = 21.7 ms.

45

Comparisons (cont’d)Fast Server v.s A Set of Slow Servers #1If we have an M/M/4 system with service rate μ=50 ppsfor each server, and another M/M/1 system with service rate 4μ = 200 pps. Both arrival rate is λ = 100 pps

delay for M/M/4 = 21.7 msdelay for M/M/1 = 10 ms

46

Comparisons (cont’d)Fast Server v.s A Set of Slow Servers #2If we have n M/M/1 system with service rate μ pps for each server, and another M/M/1 system with service rate nμ pps. Both arrival rate is nλ pps

μλ

μλ

μλ

…

nλ nμ

11/1

T μρ

=−

S1 S2

21/ 1/

11

n nT nn

μ μλ ρμ

= =−−

12

TTn

∴ =

47

M/M/m/m

Multiple Servers, No Storage(m-Server Loss Systems)

48

M/M/m/mThere are available m servers, each newly arriving customers is given a server, if a customers arrives when all servers are occupied, that customer is loste.g. telephony system.

kλ =

k kμ μ=

if k mλ <0 if k m<

49

M/M/m/m (cont’d)

kP =0

1( / ) if k m!

kPk

λ μ⋅ ≤

0 if k m>

1

00

1( / )!

k

kP

kλ μ

−∞

=

⎡ ⎤⇒ = ⎢ ⎥

⎣ ⎦∑

50

M/M/m/m (cont’d)Let pm describes the fraction of time that all mservers are busy. The name given to this probability expression is Erlang’s loss formulaand is given by

This equation is also referred to as Erlang’s B formula and is commonly denoted by B(m,λ/μ)http://www.erlang.com

0

( / ) / !

( / ) / !

m

m mk

k

mpk

λ μ

λ μ=

=

∑