1

5. An Introduction to Queueing Theory 5.1. INTRODUCTION To wait is the most frequent activity today. We are always in the same situation :

Many customers want to use a finite resource. Interesting questions are: Why we have to wait? How long we must wait? How many people will stay in the line in front of me? What can be done to improve the service? Answers are given by QUEUEING THEORY (German: Bedienungstheorie, Russian: Теория массового обслуживания) Some History

1909 - A.K. ERLANG (Danish mathematician): He first considered problems of telephone traffic congestion 1930 - F. POLLACZECK publicated papers on models with Poisson input, arbitrary service, single- and multiple-servers

In Russia: Kolmogorov, Khintchin In Sweden: Palm

1950 - queueing theory as a fashion-science, but always an applied science Further growth by modern computing technique: - new application areas - new possibilities by simulation, software packages, algorithms Today exist important applications in many areas:

- machine repair - toll booths - taxi stands - inventory control - loading and unloading of ships - scheduling patients in hospital clinics - production flow (flexible manufacturing systems) - computer field - communication networks - mobile communication systems

Aim of the present lecture - to give a short introduction into o basic models o basic methods o results o applications

5.2. EXAMPLES (1) The booking clerk:

We consider a ticket office with a single clerk. Customers arrive at random times. Let τ denote the time between the arrivals of two consecutive customers. We assume that τ is a random variable with distribution function A(.). We write τ ∼ A( . ) , where A(t):= P(τ < t) describes the probability that τ is smaller than t for t ∈ R1. In a similar way let B( . ) denote the distribution function for the random service time θ, i.e., θ ∼ B( . ).

2

Problem 1: Which is the maximum load the clerk can handle? Problem 2: Which is the average number of customers in the waiting line or in the whole system? Problem 3: Which is the expected time a customer has to spend in the ticket office? Problem 4: How much customers the clerk serves in average per time unit? (2) Harbour landing places:

Assume that in a harbour exist 3 landing places. Ships arrive at random. If all places are occupied, a ship has to wait in the roads. We further assume random times for unload and load. Then we have a similar situation as in Example (1), but with 3 „clerks“. Here we can formulate additionally a

Problem 5: What happens if there are 2 or 4 or 5 landing places? (3) Harbour with finite place in the roads:

We assume the same situation as in Example (2), but with the restriction that at most 3 ships can wait in the roads.

Summary: To model such situations we have to model

a) the arrival process, b) the service unit and the service process, c) the underlying stochastic influences.

For that in the next section we consider basic ideas and methods of probability theory that can be used for analysis of queueing problems. 5.3. PREREQUISITES FROM PROBABILITY THEORY 5.3.1. Exponential distribution A random variable (rv) X is said to be exponentially distributed with parameter λ > 0 if its distribution function FX(.) has the form

FX(t) := P(X<t) = 1 - e-λ t , t ≥ 0. We write X ∼ Exp(λ), i.e., an exponentially distributed rv is described by a single parameter. Other properties of the exponential distribution are:

• density function: fX(t) = λ e -λ t , t ≥ 0 • expectation: E(X) = 1 / λ, i.e., the expected value of an exponentially

distributed rv is equal to the reciprocal of its parameter. • variance: D2X = E(X - E(X))2 = E(X2) - [E(X)]2 = 1 / λ2

For our purpose a very important property of the exponential distribution is the so-called memoryless property. To explain that property we consider the following figure, which represents realisations of arrival moments of customers. We denote the differences of these arrival moments as interarrival times. In the figure below realisations of these interarrival times are represented by bows. Let T be a fixed time moment. We are interested now in the distribution of the time, which will elapse from T on to the next arrival moment. We denote that random time by YT and call it residual interarrival time. For given time moment T the rv

3

YT describes the remaining time to the next arrival moment under the condition that the last arrival was T time units ago. We calculate now the distribution function for YT under the assumption that for the interarrival time X holds X ∼ Exp(λ), i.e., X is assumed to be exponentially distributed with parameter λ. Instead of the FY,T(t) = P(YT<t) we calculate 1 - FY,T(t) = P(YT≥t). The event {YT≥t} is equivalent to the conditional event {X≥T+t | X≥T}. Thus, using elementary properties of conditional probabilities and the fact that for exponentially distributed X holds P(X ≥ t) = e-λ⋅t, we get P( YT ≥ t ) = P(X ≥ t + TX ≥ T) = P(X ≥ t+T ∩ X > T) / P(X ≥ T) = P(X ≥ t+T) / P(X ≥ T) = = e[-λ (t+T)] / e(-λ T) = e(-λ t) = P( X ≥ t ). With P(YT ≥ t) = P( X ≥ t) we have also P(YT < t) = P(X < t) or FY,T(t) = FX(t) for all t ≥ 0 and arbitrary T ≥ 0. Thus we have proven the following CONCLUSION: The residual interarrival time Y has the same (exponential) distribution as the (whole) interarrival time X, regardless of how long ago was the last arrival. The same holds for exponentially distributed service times: The residual service time has the same (exponential) distribution as the (whole) service time, regardless of when service was started. That fact can be interpreted that exponentially distributed random variables at any chosen time moment “forget” how long takes their actual realisation. Thus the name memoryless property is obvious. Another notion for that property is Markov property. FACT: The exponential distribution is the one and only distribution that possesses the Markov property. Consequences: Mathematical models with only exponentially distributed random variables are (as a rule) mathematical tractable: The analysis does not require a state variable indicating the time elapsed since the last event. Consequence 1: The exponential distribution is used very often. Consequence 2: The memoryless property seldom exists in practice in its pure definition. ! Use the exp. distribution with care !!!!!!!!

0 T T + t

Residual interarrival time

Time

4

5.3.2. Markov chains with continuous time We consider the following simple model: Let X(t) denote the state of a system at time moment t ≥ 0. A state can be for instance the number of customers in a service system. We denote by Z the set of states for such a system and assume that Z ⊆ { 0, 1, 2, ... }, i.e., the state set is a subset of all natural numbers (including zero). For systems with random components X(t) is a rv for each t ≥ 0. Then {X(t), t ≥ 0}is a family of random variables, which we call a stochastic process. There exist a great variety of stochastic processes. We restrict ourselves to so-called Markov chains with continuous time. By that class of stochastic processes many real systems, especially queueing systems, can be modelled. On the other hand such processes are easy to understand. A Markov chain with continuous time models the dynamics, i.e. the movement through the state set, of a stochastic system in the following way: If the system is in a state i∈Z then, independent of the past history, the system (a) stays the sojourn time Vi ∼ Exp(qi) in state i, and (b) the next state j∈Z will be chosen by transition probabilities pij, i,j ∈Z, i ≠ j, where Σ pij = 1 , i ∈ Z. (3.1) j≠i

We call a stochastic process with these two properties a Markov chain (with continuous time) and say that the process possesses the Markov property. Figuratively speaking means the Markov property that the future of a process does not depend on the past but only on the present (state). The process (or the corresponding system) possesses the memoryless property, it forgets the past. REMARK: For the expected sojourn time (see the properties for exponential distribution) holds E(Vi) = 1/qi or qi = 1 / E(Vi) = E[number of leavings of state i per time unit]. Thus the parameter qi is interpreted as the total rate out of state i or more vivid as the leaving rate for state i. With the leaving rate and the transition probabilities we get qij := qi pij (3.2) as the transition rate from state i directly to state j, i ≠ j. (3.1) and (3.2) yield qi = Σ qij for each i ∈ Z, (3.3) j≠i

i.e., the leaving rate for a state is the sum of all transition rates for that state. Thus we have the following picture for the movement (and also simulation) of a Markov chain: The process starts in an initial state, stays there a random sojourn time (which is exponentially distributed), jumps immediately to a new state (chosen in accordance with transition probabilities), stays there again a random sojourn time and so on. It is a fact: For given initial state a Markov chain is completely determined by the transition rates qij, i,j∈Z, i ≠ j. ATTENTION ! The qij ´s are not probabilities but rates, i.e., for a small time interval ∆t holds P( transition from i to j in ∆t - time) ≈ qij ∆t . At least two questions arise:

5

Question 1: How to extract the qij ´s from a practical problem? Question 2: How to use them for computing performance measures of a system? An answer to these questions will be given in the next subsection.

5.3.3 The long-run behaviour of continuous time Markov chains If we are looking for the long-run behaviour it means that we are interested for the probability that the system will be in a given state j∈Z after a long time the system was started in an initial state i∈Z. In other words, we are interested in the transition probabilities pij(t) := P( X(t) = j | X(0) = i ) for all pairs of states i, j ∈Z and t ≥ 0 . Because of practical systems as the rule exist over a long time period we in fact are looking for the limits lim pij(t). t→∞

Under very mild assumptions it holds (see TIJMS[1990]): PROPERTY 1. For each state j ∈ Z, lim pij (t) = pj exists, independently of the initial state i ∈ Z.

t→∞ In addition, pj ≥ 0, j ∈ Z, and Σ pj = 1 . (3.4) j∈Z

PROPERTY 2. The limiting probabilities {pj, j ∈ Z} can be uniquely defined by the following system of linear equations: qj pj = Σ pi qij , j ∈ Z; (3.5) i≠j

Σ pj = 1. (3.6) j∈Z

INTERPRETATION of the limiting probabilities: 1. pj represents the probability that an outside observer entering the system after a long time it has been started will find the system in state j. Therefore pj can be understood as the long-run fraction of time the system will spend in state j, independently of the starting state. This interpretation explains that the limiting probabilities {pj, j ∈ Z} are called steady-state probabilities for a given Markov chain. Furthermore, if all pj ´s are positive we say that a steady-state regime exists for the chain or the underlying system. 2. For better understanding of that interpretation we consider an example. If a room is equipped with a thermostat, which should regulate the room temperature around a target level of 20° Celsius. Then independently of the initial temperature in the room after some time the thermostat has stabilized the temperature around the control level, i.e., the room respectively the temperature process has reached his steady-state regime. Steady-state regime means: The influence of the initial state is lost. Steady-state regime does not mean: The temperature is constant without random fluctuations. Such fluctuations still exist because of may be sun is shining or many people are in the room or the window is opened and so on. Steady-state regime means: System characteristics (performance measures) as well as process laws don´t depend on time, they all can be assumed to be time independent.

6

3. To explain Property 2 we can use the so-called Markov graph (see the figure below). The marked squares represent the states of the chain and the directed arcs possible transitions between states. The marks of the arcs are the corresponding transition rates. Thus the Markov graph q0k q1k qik qkj q01 qki qij 0 1 ... i j ... qjl contains all information on the considered Markov chain. Now imagine the squares as pots and the arcs as pipes. If we assume that at the beginning one liter water is allocated over all pots and that water flows out of pot i directly to pot j with rate qij then, if steady-state regime exists, the water flow comes after some time to an equilibrium. We can say that in the steady-state regime holds the following balance principle:

For any state the input stream is equal to the output-stream .

If we now consider the steady-state probability pj as the amount of water, which on average over a long time is allocated to pot j, the equations (3.5) can be understood as balance or equilibrium equations: The output stream of water is equal to pj ⋅ qj, whereas the input stream from all other pots i≠j equals Σ pi ⋅ qij. Equation (3.6), the normalizing equation, says only i≠j

that the amount of water respectively probabilities of all pots or states is equal to one. Thus we have a partial answer to Question 2: The transition rates qij are sufficient to define the steady-state probabilities {pj, j ∈ Z} from which we can deduce all interesting performance measures for a system in the steady-state regime. Answer to Question 1 gives an algorithm, which we call Markov chain method (MCM). For better understanding of what we have to do applying MCM we consider the following simple example. Example Reliability of a control system Let us assume that 2 computers control independently of each other the unloading of an oil tanker. Each computer will fail after a random time T. There is a single repairman who repairs a failed computer. The repairman at the same time can repair only one computer. The repair time R is also an rv. We assume that T ∼ Exp(λ) and R ∼ Exp(µ). Thus the mean time to failure is E(T) = 1/λ and the mean repair time is E(R) = 1/µ. We denote the parameters λ and µ as failure rate respectively repair rate. The problem we have to solve is to calculate for given failure and repair rates the availability of the control system in steady-state regime. Obviously, the system is available if at least one computer is at work. Before we solve our problem we formulate the MCM by the following four steps:

7

Step1: Definition of the states (state set Z). To realise step 1, the most difficult step, we can use the following simple rule: Various states of the system (Markov chain) differ only by number and type of

running exponential distributed random variables. Step 2: Definition of the intensities according to the rule: The value for qi is equal to the sum of the parameters of all random variables which (i) are running in state i and which (ii) separate run out leads to the event that state i will be left. The value for qij is equal to the sum of the parameters of all random variables which (i) are running in state i and which (ii) separate run out leads to the transition i →j. ATTENTION: After step 2 check whether qi = Σ qij ! j≠i

Step 3: Derivation of system (3.5) of equations (for the steady-state probabilities) by applying the balance principle to all states of the Markov graph. Step 4: Solution of (3.5) and (3.6) We compute the steady-state probabilities and use them to define interesting

performance measures. Now we apply these steps to our problem “Reliability of a control system”. Step 1: We have three states, which we denote as below: 0 - no computer is failed (2 rvs T are running) 1 - 1 computer is failed (T and R are running) 2 - 2 computers are failed (one R is running; the second failed computer is waiting for repair) Step 2: We get q0 = 2 ⋅ λ ; q01 = 2 ⋅ λ ; q02 = 0 ; q1 = λ + µ ; q10 = µ ; q12 = λ ; q2 = µ ; q20 = 0 ; q21 = µ and the following Markov graph

2 λ λ µ µ We remark that the check after step 2 is okay. Step 3: Using the balance principle we get from the Markov graph 2λ ⋅ p0 = µ ⋅ p1 ( λ+µ) ⋅ p1 = 2λ ⋅ p0 + µ ⋅ p2 µ ⋅ p2 = λ ⋅ p1 Step 4: Together with the normalizing equation p0 + p1 + p2 = 1 we calculate

p0 = µ2 / [( µ + λ )2 + λ2 ] = E(T)2 / [ E(T)2 + 2E(T)E(R) + 2E(R)2

p1 = 2 λ µ / [(µ+λ)2+ λ2] = 2E(T)E(R) / [E(T)2+2E(T)E(R)+2E(R)2

p2 = 2 λ2 / [ ( µ+λ )2+ λ2] = 2 E(R)2 / [E(T)2+2E(T)E(R)+2E(R)2

Now we can calculate the steady-state availability of the system as

0 1 2

8

Kavail = p0 + p1 = (µ2 + 2 λµ) / [ ( µ+λ )2+ λ2]. If we assume that λ = 0.1 h-1 and µ = 0.8 h-1, i. e., the average time to failure is 10 h and the average repair time is equal to 1 ¼ h then we get p0 = 64/82 ≈ 0.7805; p1 = 16/82 ≈ 0.1951; p2 = 2/82 ≈ 0.0244 and

Kavail = 80/82 ≈ 0.9756 One question remains: Why we could apply MCM? To answer that important question we state the following fact on the applicability of the MCM:

MCM is applicable if Assumption EXP is fulfilled,

where Assumption EXP means that all random variables describing the behaviour of the elements of the system are exponential distributed random variables.

We mention that the exponential distribution possesses the memoryless or Markov property. Thus from Assumption EXP follows that the behaviour of the corresponding system can be modelled as a Markov chain with continuous time. In the next subchapter we solve by MCM a more realistic example. 5.3.4. MCM and the booking clerk The first example in chapter 5.2 describes the service of randomly arriving customers by a single booking clerk. The random influences came from two rvs – the interarrival times τ of customers and their service times θ by the clerk. The corresponding distribution functions we denoted by A(.) and B(.). If both distributions are assumed to be exponential Assumption EXP is fulfilled and we can apply MCM. Therefore we assume A(t) = 1 - exp(-λ t) , t≥0, λ>0, and B(t) = 1 - exp(-µ t) , t≥0, µ>0. From the properties of the exponential distribution we know that E(τ) = 1/λ and E(θ) = 1/µ or vice versa λ = 1/E(τ) and µ = 1/E(θ). From the last equalities follows that λ can be interpreted as arrival intensity or arrival rate and µ as service intensity respectively rate. Intensities/rates are fixed or given constants with measuring unit number of customers per time unit, i.e. 1/(time unit). The most important question in Chapter 2 was formulated as Problem 1: Which is the maximum load the clerk can handle? The load is given by the arrival rate. When the clerk is overloaded the number of waiting customers will increase and increase. In such a case all steady-state probabilities will be zero and no steady-state regime will exist. Therefore, to solve Problem 1 is equivalent to answer the question “When exists steady-state regime?”. To find corresponding conditions we simply assume that steady-state regime exists, we calculate the steady-state probabilities, and find conditions under which these probabilities will be positive. Let us apply now MCM. Step 1: State definition Obviously, the number of customers in the system characterises the behaviour of the service system. Thus we have Z = {0, 1, 2, ...} Step 2: Definition of the intensities/rates We consider the states one after another. In state 0 only an interarrival time is running. The completion of that time means that a customer arrives, i.e., the system moves to state 1. This happens with intensity λ. In all other states i ≥ 1 two rvs are running – an interarrival time and a service time. Completion of the interarrival time results in an arrival of a customer and a transition to state i+1, whereas completion of a service time results in the departure of the served customer and a transition to state i-1. Thus we get for state 0 the rates q0 = λ and q01 = λ, and for all states i≥1 the rates qi = (λ+µ), qii-1 = µ, and qii+1 = λ. All this information we represent in the following Markov graph:

9

λ λ λ λ λ µ µ µ µ µ Step 3: Using the balance principle we derive the equation system (3.5) from the Markov graph as λ p0 = µ p1 ( λ + µ ) pi = λ pi-1 + µ pi+1 , i ≥1 Step 4: In Step 3 we have an infinite number of variables pi and an infinite number of equations. One may think that a solution is not possible. But the Markov graph shows a nice structure – direct transitions are possible only to neighbouring states. Such equation systems are typical for queueing systems. For all of them we can apply a simple solution approach: We choose one state, mostly state 0, and express all unknown variables pi for i ≥1 as functions of p0. Replacing in the normalizing equation (3.6) all variables pi by their corresponding function of p0 we get a single equation for p0, which we can solve. Finally, we put the resulting value for p0 into the functions for pi and get the result. It is convenient to introduce a constant r as r = λ / µ. From the first equation in Step 3 we get p1 = r . p0, i.e., p1 is a very simple function of p0. Next we consider the equation in Step 3 for i = 1. We get ( λ + µ ) p1 = λ p0 + µ p2, which, using the first equation λ p0 = µ p1, can be reduced to λ p1 = µ p2 or p2 = r p1 = r2 p0. In the last equation we have used that p1 = r p0. By induction we can prove that for all states n≥1 holds pn = rn p0. Now we use the normalizing equation (3.6), which yields ∞ ∞ ∞ 1 = Σ pn = p0 Σ rn or p0 = 1 / Σ rn n=0 n=0 n=0 Steady-state regime exists if and only if all steady-state probabilities are positive. Obviously, ∞ p0 > 0 is equivalent to the condition r < 1. In this case Σ rn = 1 / (1 – r). n=0

Summarising we get the following results:

I. Steady-state regime exists if and only if r < 1 or λ < µ, i.e., the load must be strongly smaller than the capacity of the clerk (the service system).

II. For λ < µ we have p0 = 1 - r and pn = rn (1 - r) , n > 0.

From the equation r = 1 - p0 = P( clerk is busy) we get a nice interpretation for the constant r: r is called utilization factor or server utilization.

III. Let L denote the expected number of customers in the system. With the formulas in II. we calculate ∞ ∞ L = Σ n pn = Σ n rn (1 - r) = r / (1 - r) = λ/(µ-λ) n=0 n=1

In an analogous way we derive for the expected number Q of customers waiting in the line that

∞

Q = Σ (n-1) pn = r2 / (1 - r). n=1

0 1 i- 1 i i +1

10

Remark: Q = L - 1 . P(server is busy) = L – 1. r = r2 / ( 1 - r ) The table below shows the L and Q values for some server utilizations:

r 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.95 0.99 0.995 L .1111 .2500 .4286 .6666 1.000 1.500 2.333 4.000 9.000 19.00 99.00 199.0 Q .0111 .0500 .1286 .2666 0.500 0.900 1.633 3.200 8.100 18.05 98.01 198.0From that simple example we derive an important CONSEQUENCE for practice: HIGH UTILIZATION of server systems is always equivalent to GREAT WAITING LINES (and , consequently, to high waiting times). L 9 4 2 1 0 0.5 0.6 0.7 0.8 0.9 1 r Graph for L as function of r 5.4. BASIC NOTIONS AND CLASSICAL MODELS Classical queueing models investigate the following situation:

• At random moments jobs or customers or clients arrive in a service station, • wait in a queue until their service will be started, • will be served a random time, and • leave the system after service completion.

For a corresponding model we have to formalize the following parts. a) The arrival pattern (the arrival process). In all here considered models we assume that - arrivals come one at a time: 0 = t0 < t1 < t2 < ... < tn < ..., where tn denotes the arrival time of the n-th customer; - τi := ti+1 - ti represents the interarrival time for job i, i ≥ 1

11

- ASSUMPTION for the arrival process: {τi, i≥1 } is a sequence of i.i.d. (independent and identical distributed) random

variables with distribution function A( . ), i.e., A(t) := P( τ1 < t ), t≥0. It holds E(τ1) < ∞ . - α:= 1 / E(τ1) is called arrival rate (e.g., for A(t)=1-exp(-λt) holds α=λ). b) The queue. First, there may be various queueing disciplines how the server chooses waiting jobs from the queue. The most popular ones are FIFO (first in first out), LIFO (last in first out), random, with various priorities and others. Here we consider only FIFO, where jobs are served in the same sequence as they arrived. A second characteristic for the queue is the waiting line capacity. We distinguish between lost systems (no waiting line), waiting systems (infinite waiting capacity), and finite systems (finite waiting capacity). c) The service pattern. There exist single-server-, multi-server-, infinite-server-systems. We assume that - all servers work independent of each other and that they are „identical“ with respect to the service time distribution; - one server serves one job at a time; - θi denote the service time for job i, i≥1 - ASSUMPTION for the service process: {θi, i≥1 } is a sequence of i.i.d. random variables with distribution function B( . ), i.e., B(t):= P( θ1<t), t≥0. It holds E(θ1) < ∞ .

- β := 1 / E(θ1) is called service rate (of a single server) (e.g., for B(t)=1-exp(-µt) we have β = µ). To abbreviate the description of queueing models is used the so-called KENDALL´s code. We consider the variant with four elements A/B/s/n, where A, B describe the arrival and service process, e.g., M - Exponential distribution (Markov property), D - Deterministic process, or G - General distribution;

s ≥ 1 and n ≥ 0 are integers, which denote the number of servers respectively the number of waiting places. For instance the booking clerk is an example for an M/M/1/∞ - model. The most important problems investigated in queueing theory are answers to the following questions. 1. Under which conditions does exist a steady-state regime? 2. Definition of probabilities, distribution functions, and expectations. 3. Comparison and optimization of queueing systems (according to various criteria). Here we consider all models under the following main

ASSUMPTION: We always consider the systems in steady-state.

Interesting performance measures (for the steady-state) are (I) Probabilities for defined states as

p(0) – the probability that the system is empty,

12

p(k) – the probability that k jobs are in the system, pw - the probability that an arriving job has to wait, pv - the probability that an arriving job is lost.

(II) Distribution functions (d.f.) for defined random variables as W( . ) - d.f. for the waiting time,

V( . ) - d.f. for the total time a job spends in the system (the response or sojourn time).

(III) Expectations for defined random variables as ∞

L := Σ k p(k) - the expected number of jobs in the system k=1

∞

Q := Σ k p(k+s) - the expected number of jobs in the queue k=1

s ∞ S := Σ k p(k) + s Σ p(k) - the expected number of busy servers k=1 k=s+1

W, V - expected waiting time, expected sojourn time

There exist some important relations: V = W + E( θ1 ) (4.1) L = Q + S (4.2)

A broad application finds LITTLE´s law in two forms: and Q = α ∗ W From this and (4.2) follows S = α / β = α ∗ E( θ1 ) In these laws the most important performance measures of a queueing system are related to each other respectively to the input data arrival and service intensity of the system.

With respect to the Existence of steady-state regime we have some general results. (a) For G/G/s/∞ - models it exists if and only if

E( θ1 ) < s ∗ E( τ1 ) or α < s ∗ β (4.3) (b) For G/G/s/n - models with n < ∞ it always exists. Remark: Condition (4.3) means that the arrival rate should be lower than the sum of service rates of all servers or equivalently that the offered load should be lower than the service capacities. Next we give some helpful formulas for models, which are important for us.

L = α · V

13

(A) The M/M/1/∞ - model 1. Steady-state regime exists iff r = λ / µ < 1. For the steady-state regime holds: 2. p(0) = 1 - r p(k) = rk (1 - r) , k=1, 2, ... pw = 1 - p(0) = r 3. L = r / (1 - r) Q = r2 / (1 - r) S = r W = Q / λ = r / ( µ - λ ) V = L / λ = 1 / ( µ - λ ) = W + 1 / µ = W / r (B) The M/M/s/∞ - model For Example 2 in Chapter 5.2 („Harbour landing places“) we assume s = 3, n = ∞, and A(t) = 1 - exp(- λt), t ≥ 0, λ > 0, respectively B(t) = 1 - exp(- µt), t ≥ 0, µ > 0. In accordance with the Kendall-code we have an M/M/3/∞ - model. For an arbitrary number s of landing places we get an M/M/s/∞ - model. Obviously, Assumption EXP is fulfilled. Thus we can apply MCM. Step 1: Again as a state we take the number of jobs in the system. Thus we get Z = {0, 1, 2, ... }.

Step 2: Instead of the definition of intensities we develop the corresponding Markov graph, which in the present example is given below.

λ λ λ λ λ λ µ 2µ 3µ (s-1)µ s µ s µ sµ

Step 3: Application of the balance principle to derive the system (3.5) of equations. We get λ p(0) = µ p(1) (λ + µ) p(1) = λ p(0) + 2 µ p(2) [λ+(s-1)µ] p(s-1) = λ p(s-2) + s µ p(s) (λ+sµ) p(k) = λ p(k) + s µ p(k+1) , k =s, s+1, ...

Step 4: Solution of (3.5) and the normalizing equation (3.6), using the notion r = λ / µ, yields s

p(0) = 1 / { Σ rk / k! + rs+1 / [ s! (s-r)] } (4.4) k=0

1 / k! , k = 1(1)s; p(k) = p(0) rk (4.5) 1 / ( s! sk-s) , k > s; Other results are:

• steady-state regime exists iff r < s or λ < µ . s .

0 1 2 s-1 s s+1

14

• pw = p(s) · s · µ / (s · µ - λ) (4.6) • Q = r · pw / /s - r) = p(s) · s · r /( s - r )2 (4.7) • S = r • L = Q + S = Q + r (4.8) • W = Q / λ = pw / (s · µ - λ) (4.9)

With the above formulas we can answer an important for practice question. QUESTION: Which investment is better - to install 3 new additional landing places or to modernize the 3 existing places? Here we answer the simplified problem: What is better - two slow servers (with service rate µ) or a single quick server (with service rate 2µ)? To get an answer we compare the following 3 variants. Variant 1: λ = 6 ships per day, µ = 4 ships per day, s = 2, one common waiting line. Variant 2: λ = 6 ships per day, µ = 8 ships per day, s = 1. Variant 3: Two independent service stations with own waiting line, λ = 3 ships per day and µ = 4 ships per day. Variant 1 can be modelled as an M/M/2/∞ - model with r = λ / µ = 1.5. From equ. (4.4) - (4.8) we compute: p(1)(0) = 1 / { 1 + 1.5 + 2.25/2 + 1.53/[2*(2-1.5)]} = 1 / 7 = 0.142857 p(1)(2) = p(1)(0) · r2 / 2 = p(1)(0) · 1.125 = 0.160714 p(1)

w = p(1)(2) · 2 · 4 / (8 - 6) = 9/14 = 0.642857 Q(1) = r · p(1)

w / (2 - r) = 1.5 · 9 / (14 · 0.5) = 27/14 = 1.928571 L(1) = Q(1) + r = 3.428571 Variant 2 is an M/M/1/∞ - model with r = 0.75. The corresponding formulas yield:

p(2)(0) = 1 - r = 0.25

p(2)w = 1 - p(2)(0) = 0.75

Q(2) = r2 / (1 - r) = 2.25

L(2) = r / (1 - r) = 3.00

Variant 3 leads to two isolated M/M/1/∞ - models with r = 0.75.

15

Then we have for the i-th model, i=1, 2: Li = 3 and Qi = 2.25 and for the whole system L(3) = 6 and Q(3)= 4.5.

What can we conclude from these results?

Conclusion 1. The union of two independent service systems (Variant 3) to one system with common waiting line (Variant 1) leads to a substantial improvement of service quality. Thus for the above numerical example L(1) amounts only 57.14 % of L(3) and Q(1) amounts only 42.86 % of Q(3) .

Conclusion 2. The replacement of 2 slow servers (Variant 1) by one fast server (Variant 2) leads to contradictory results. For the considered example L(2) amounts only 87.5 % of L(1) , but Q(2) amounts 116.67 % of Q(1) .

Which variant to prefer? Without additional assumptions, e.g., some gain and cost structure (for the steady-state regime) an answer is not possible. For that let denote cs - service cost per time unit (hour, day,...) and per server, cw - waiting cost per time unit and per waiting customer, g - gain per served customer . Then we can consider the criterion function

G(s) = g · λ - [ s · cs + cw · Q(s) ]. (4.10)

G(s) represents the expected income per time unit in the steady-state regime for a M/M/s/∞ - model. Remark: For maximization of G we have to minimize the expected cost part C(s) := s · cs + cw · Q(s). (4.11) Therefore, the optimal number s* of servers is influenced only by the cost factors for the servers and for waiting customers. The optimum must be a compromise. We solve the problem for some relations of these two cost parts. (i) cs = cw / 2: In this case we calculate from (4.11) that C(1)(2) = 2·cs + 2·cs·Q(1)(2) = 2·cs·41/14 = 5.8571 cs and C(2)(1) = 1·cs + 2·cs·Q(2)(1) = 5.5000 cs. Thus variant 2, the single fast server, is advantageous. (ii) cs = cw / 4: Now we get C(1)(2) = 9.71 cs and C(2)(1) = 10.00 cs. Here Variant 1 is better. Conclusion: The answer which variant outperforms the other one depends on the relation of corresponding cost parameters. Let us briefly investigate another question: Which is the optimal number s* of servers? From equ. (4.3) we know that steady-state regime exists for an M/M/s/∞ - model iff s > α/β or s > λ/µ. The answer, naturally, depends on what means „optimal“. We consider two simple criterion functions.

16

A) The optimal (minimal) number of servers to reach a given expected sojourn time. To be more concrete we assume that: - ships arrive at a rate of λ = 3.5 ships/day, - service rate of an unload/load station is µ = 1 ship/day, and we have - to define an s* : V(s*) ≤ 36 hours SOLUTION: With the assumed data we have r = λ / µ, i.e., at least s*≥ 4 must be. We start with s=4: p(0) = 1 / [ 1 + 3.5 + 3.52/2 + 3.53/6 + 3.54/24 + 3.55/(24*0.5)] = 1 / 67.791666 = 0.014751 p(4) = p(0)*3.54/42 = 0.092233 Q(4) = p(4)*4*3.5/(4 - 3.5)2 = 5.165028 ships W(4) = Q(4) / 3.5 = 1.4757 days V(4) = W(4) + 1 / µ = 2.4757 days = 59.42 hours We see that the goal 36 h can not be reached with s = 4.

For s=5 we get: p(0) = 0.025898 p(5) = 0.113351 Q(5) = 0.881622 ships V(5) = 1.25189 days ≈ 30.05 h Thus the solution suggests to install 5 unload/load stations. Then the expected sojourn time of a ship in the harbour will be approximately 30 hours.

B) The cost/gain-optimal number of servers as above. We have G(s) = g·λ - [ s · cs + cw · Q(s) ] , s ≥ λ / µ, and C(s) = s·cs + cw·Q(s). Let λ = 6 ships/day and µ = 4 ships/day. (Variant 1)

a) Case cs = cw/2. For different number s we calculate C(2) = cs· [2 + 2·1.928571] ≈ 5.857 cs C(3) = cs· [3 + 2·0.236842] ≈ 3.474 cs C(4) = cs· [4 + 2· { >0} ] > 4.000 cs

Thus s* = 3 is optimal and the maximal expected gain equals G(3) = 6 · g - 3.474 cs.

b) Case cs = 2⋅cw. In a similar manner we get C(2) = cs ⋅ [2 + 0.5 ⋅ 1.928571] ≈ 2.964 cs C(3) = cs ⋅ [3 + 0.5 ⋅ 0.236842] ≈ 3.118 cs

Such that s* = 2. c) Case cs = cw/8. Here we derive C(2) = cs * [2 + 8 * 1.928571] ≈ 17.429 cs

C(3) = cs * [3 + 8 * 0.236842] ≈ 4.895 cs C(4) = cs * [4 + 8 * 0.044751] ≈ 4.358 cs C(5) = cs * [5 + 8 * { > 0 } ] > 5 cs and s* = 4. CONCLUSION 3: The maximization of the goal function (4.10) can be based on two properties: (i) If cost factor cs increases then the optimal solution s* decreases.

17

(ii) If cost factor cw increases then the optimal solution s* increases too. SUMMARY: The MCM-approach is * very simple, * very usefull and has a * wide range of applications. However: A harbour is a finite area, i.e., there is a finite number of waiting places. Up to now we have assumed infinite many waiting places. In practice however often the waiting capacity is finite. Some models of such kind will be considered in the last chapter. 5.5. Finite service systems We have a finite service system if there are finite many servers and waiting places, i.e., the maximum number of customers in the system is a finite positive integer, say S. Obviously, steady-state regime always exists for finite systems because of these systems have only a finite state set. 5.5.1. The M/M/1/S-1 model In this model we have a single server and S-1 waiting places. If all waiting places are busy an arriving customer will be rejected or lost. We assume exponentially distributed interarrival times and service times with distribution functions A(t) = 1 - exp(-λ t), t ≥ 0, λ > 0, and B(t) = 1 - exp(-µ t), t ≥ 0, µ > 0, respectively. Because of Assumption EXP is fulfilled we can apply MCM. Step 1: As usual we define as the state of the system the number of customers in system. Thus we have Z = {0, 1, 2, ..., S}. Step 2: The corresponding Markov graph is given below. λ λ λ λ λ µ µ µ µ µ Step 3: Again, putting r = λ/µ, we derive the system (3.5) of equations by applying the balance principle.

λ p(0) = µ p(1) ( λ + µ) p(i) = λ p(i-1) + µ p(i+1) , i=1..S-1 µ p(S) = λ p(S-1)

Step 4: Solution of (3.5) together with the normalizing equation (3.6). From the first equation in step 1 we get p(1) = r · p(0). It is easy to see that using the result for the (i-1)-st equation we get from the i-th equation that p(i+1) = r · p(i) = r2 · p(i-1) = ...= ri · p(0), i=1..S-1. Putting this into the normalizing equation we have p(0) = 1 / (1 + r + r2 + ... + rS). Now we have to distinguish between two cases. Case r = 1: Here we get p(i) = 1 / (S+1) for i=0..S, i.e., all steady-state probabilities have the same probability. S

Case r ≠ 1: Then, using the equality Σ ri = (1 - rS+1) / (1 - r), we calculate i=0

0 1 2 i S-1 S

18

p(0) = (1 - r) / (1 - rS+1) (5.1) and p(i) = ri (1 - r) / (1 - rS+1), i=1..S. (5.2) Once more we remark: ! Steady-state regime exists without any conditions on the relation of λ and µ!!! The calculation of performance measures is easy for case r = 1:

For S ≥ 1 we get L(S) = S/2 and Q(S) = S/2 · (S-1) / (S+1). Case r ≠ 1: S S 1) L(S) = Σ i·p(i) = (1 - r) / (1 - rS+1) · Σ i · ri = = r / (1 - r) ·[1 - (S+1) rS + S rS+1 ]/(1 - r S+1)

i=1 i=1 S

2) Q(S) = Σ (i-1) p(i) = L(S) - [1-p(0)] = L - (r-rS+1) / (1-rS+1) = i=1

= r / (1- r) [1- (S+1)·rS + S·rS+1 - (1-rS)·(1-r)] / (1- rS+1) = r2 / (1 - r) [ 1 -S·rS-1+(S-1)·rS] / (1 - rS+1) 3) P( rejection of a customer) = P(all places are occupied) = p(S) 4) Intensity of the offered load : λ Rate of rejected parts: λ p(S) Rate of accepted load: λ [1 - p(S)] An interesting problem is to ask “Which is the optimal S?”. For this we need a criterion. We take the criterion “Maximum of expected reward per time unit in the steady-state regime. As gain and cost structure we assume the following: g - gain per served customer w - waiting cost per waiting customer per time unit a - rejection cost per rejected customer Let G(S) denote the expected reward earned from the M/M/1/S-1 model per time unit in the steady-state regime. Then with the assumed gain and cost structure we have

G(S) = λ [ 1 - p(S) ] g - w Q(S) - a·λ·p(S) = λ · g - [ λ· p(S)·(a + g) + w·Q(S)]. The optimisation problem is to define an S* such that G(S*) ≥ G(S) for S ≥ 1. To formulate a solution algorithm for that problem we use the following PROPERTY 5.1. (i) For 0 < r < 1 it holds :

Q( S ) is an increasing and concave function of S. p( S ) is a decreasing and convex function of S.

(ii) For r ≥ 1 it holds : Q(S) is an increasing and convex function of S. p(S) is a decreasing and convex function of S.

19

From Property 5.1 follows that for all r ≥1 the criterion function G(S) is a concave function of S. From the concavity property of the criterion function we get the following Optimization algorithm: (for r ≥ 1)

1. Choose an initial value S(0) and compute G(0) = G(S(0)). 2. S := S(0) + 1. 3. If G(S) > G(0)

then begin repeat S := S + 1 until G(S)<G(S-1); S* := S-1; end else begin repeat S := S - 1 until G(S)< G(S+1) S* :=S +1 end.

The idea of the algorithm is simple. We choose an arbitrary starting value for S and calculate the corresponding gain. Now we increase S by 1. If we get a higher gain for the new S+1 then we are moving in the right direction to the optima (this follows from the concavity property). Otherwise we have to move into the opposite direction, i.e., we must decrease the initial S. 5.5.2. The multi-server lost system M/M/s/0 Typical characteristics of the M/M/s/0 model are exponentially distributed interarrival and service times with parameters λ respectively µ, s ≥ 1 identical servers, and the fact that an arriving customer is lost if all servers are busy. Again, steady-state regime exists in all cases. A call centre is a typical situation, which can be modeled by an M/M/s/0 lost system. Another example is the following. Example 5.1 Given: A small logistic company with s = 3 carriers. Jobs arrive with exponentially distributed interarrival times (let the parameter be λ > 0). If no carrier is free the arriving job is rejected. A transportation job needs an exponentially distributed (service) time with parameter µ > 0. Demanded: a) Which is the average utilization of a carrier when service time is 24 hours?

b) Which is the probability that an arriving job is rejected? Solution Obviously, MCM is applicable. Let r = λ/µ. Here we define now the state as the number of jobs in the system, which is equivalent to the number of carriers in use. The corresponding Markov graph is λ λ λ 0 1 2 3 µ 2µ 3µ In the typical way (see examples above) we calculate

p1 = r ⋅ p0, p2 = r2/2! ⋅ p0, p3 = r3/3! ⋅ p0, and p0 = [1 + r + r2/2! + r3/3!]-1.

20

Thus we can answer both questions from above. a) The average utilization U of a single carrier is equal to U = 1/3 p1 + 2/3 p2 + 3/3 p3. To

explain that result we notice that utilization means the average portion of time the carrier is busy (in steady-state regime). With probability p1 one carrier is busy, and this portion of time we have to allocate equally to all three available carriers. The similar argument holds for states 2 and 3.

b) An arriving job is lost if and only if all carriers are busy. Therefore the probability PV that a job is lost is equal to PV = P(job is lost) = p3.

Let us calculate a numerical example with s = 3, λ = 12 jobs per day = 12 d-1, and an expected service time E(service time for a single transportation job) = 6 h. From Chapter 2 we get µ = 1/(6 h) = 4 d-1. Thus we have r = 3. With the above derived formulas we calculate

p0 = 1 / [1 + 3 + 9/2 + 9/2] = 1/13, p1 = 3/12 , and p2 = p3 = 9/26. Consequently, we get

U = 1/13 + 3/13 + 9/26 = 17/26 ≈ 0,65385 and

PV = 9/26 ≈ 0,34615. It is easy to generalize the derived formulas to arbitrary number of servers. Here we give formulas for some important performance measures for the M/M/s/0 model with arbitrary s ≥ 1 in the steady-state regime. a) Traffic intensity r = λ / µ b) Steady-state probabilities

1

00 !

)(−

=

== ∑

s

k

k

kremptyissystemPp

skpkrsysteminarecustomerskPp

k

k ...0,!

)( 0 =⋅== (Erlang’s formula)

pk = 0 für k > s

c) Lost probability PV = ps d) Arrival rate of accepted customers λA = λ⋅[1 – ps] e) Utilization (of a single server) U = λA / (s⋅µ) f) Expected number of busy servers = expected number of customers in system S = L = λA / µ = r ⋅ [1 – ps] = U ⋅ s = r (1 – PV) g) Expected times V = 1 / µ and W = 0. Example 5.1 – Continuation We have now an idea: To improve the utilization of carriers we register maximum two transportation jobs in case all carriers are in the way. Thus we get an M/M/s/n – model with s = 3 and n = 2.

21

5.5.3. The multi-server system M/M/s/n. We investigate now the continuation of Example 5.1, i.e., we derivate formulas for the M/M/3/2 system. To take the number of used carriers as a state is not sufficient because of in the case all carriers are in use we have three possibilities – no customer is waiting, a single one or two are waiting. Therefore we define the state as the number of customers in the system. The Markov graph now is shown below. λ λ λ λ λ 0 1 2 3 4 5 µ 2µ 3µ 3µ 3µ From this graph, applying again the balance principle, we finally get the following equations for the steady-state probabilities:

p1 = r ⋅ p0, p2 = r2/2! ⋅ p0, p3 = r3/3! ⋅ p0, p4 = r/3 ⋅ p3 = r/3 ⋅ r3/3! ⋅ p0, p5 = r/3 ⋅ p4 = (r/3)2 ⋅ r3/3! ⋅ p0, and p0 = [1 + r + r2/2! + r3/3! + r/3 ⋅ r3/3! + (r/3)2 ⋅ r3/3!]-1.

The two performance measures that were ask for are U = 1/3 p1 + 2/3 p2 + 3/3 (p3 + p4 + p5 )

and PV = P(job is lost) = p5.

For a numerical example we take the same data as in 5.5.2, i.e., we assume s = 3, λ = 12day-1, and E(service time for a single transportation job) = 6 h. Again we have r = 3. From the above given formulas we calculate

p0 = 1 / [1 + 3 + 9/2 + 9/2 + 9/2⋅(1 + 1)] = 1/22 ≈ 0.04545, p1 = 3/22, p2 = p3 = 9/44, p4 = p5 = 9/44, U = 1/3 ⋅ [(6 + 18 + 27)/44 + 18/44] = 35/44 ≈ 0,7954, and PV = 9/44 ≈ 0,2045.

We remark that the interesting performance measures of the M/M/3/2 model in comparison to the M/M/3/0 model are improved by allowing two waiting places:

The utilization increases from 0,65385 to 0,7954 and the lost probability decreases from 0,34615 to 0,2045.

In the same way we can calculate corresponding formulas for the general M/M/s/n model with s ≥ 1 and 1 ≤ n < ∞. a) Traffic intensity r = λ / µ b) Steady-state probabilities

1

100 !!

−

==

+= ∑∑

n

k

kss

k

k

sr

sr

krp

=

⋅

=⋅= −

Ksksr

sr

skkr

pp sks

k

k

...,!

...0,!

0

pk = 0 für k > s

22

c) Lost probability PV = ps+n d) Arrival rate of accepted customers λA = λ⋅[1 – ps+n] e) Utilization (of a single server) U = λA / (s⋅µ)

f) Expected number of customers in system ∑+

=

⋅=ns

kkpkL

1

g) Expected number of busy servers = expected number of customers in service

∑ ⋅−−=

∑−⋅∑ +⋅=µλ=−

=

−

=

−

=

1

1

1

0

1

01

s

kk

s

kk

s

kkA p)ks(spspk/S

h) Expected number of waiting customers ∑ ⋅−=+=

K

skkp)sk(Q

1, which in case r ≠ s is

equal to [ ]nns

aanaasaprQ )1)(1(1)1(!

12

0 −+−−−

= + , where a = r/s.

i) Expected waiting time W = Q / λA

j) pService = P(customer is served without waiting) = ∑−

=

1s

0kkp

k) pWait = P(customer hast o wait) = ∑−+

=

1ns

skkp

l) Expected sojourn time V = L / λA Special case s = 1 For abbreviation we denote K = s+n. a) r ≠ 1 p0 = (1–r) / (1–rK+1) pk = rk p0 , k=0, 1, …, K PV = (1–r)⋅ rK / (1–rK+1) L = r / (1–r) – r / (1–r) (K + 1) pK = r / (1–r) – rK+1 / (1–rK+1) (K + 1) S = 1 – p0 = r (1-pK)

Q = L – S = r2 / (1-r) – r (K+r) / (1-r) pK b) r = 1 pk = 1 / (K+1) , k=0, 1, …, K L = K / 2 Q = K/2 ⋅ (K-1)/(K+1)