Queuing TheoryFor Dummies

Jean‐YvesLeBoudec1

All You Need to Know About Queuing Theory

Queuingisessentialtounderstandthebehaviour ofcomplexcomputerandcommunicationsystemsIndepthanalysisofqueuingsystemsishardFortunately,themostimportantresultsareeasy

Wewillstudyonlysimpleconcepts

2

1. Deterministic QueuingEasybutpowerfulAppliestodeterministicandtransientanalysisExample:playbackbuffersizing

3

Use of Cumulative Functions

4

Solution of Playback Delay Pb

5

A(t) A’(t) D(t)

time

bits

d(0)d(0) - d(0) +

d(t)

A.

2. Operational Laws

Intuition:SayeverycustomerpaysoneFrperminutepresentPayoffpercustomer=RRateatwhichwereceivemoney=NInaverageλcustomersperminute,N=λR

6

Little Again

ConsiderasimulationwhereyoumeasureRandN.YouusetwocountersresponseTimeCtr andqueueLengthCtr. Atendofsimulation,estimate

R= responseTimeCtr /NbCustN=queueLengthCtr /T

whereNbCust =numberofcustomersservedandT=simulationduration

BothresponseTimeCtr=0 andqueueLengthCtr=0 initiallyQ:Whenanarrivalordepartureeventoccurs,howarebothcountersupdated?A: queueLengthCtr +=(tnew ‐ told). q(told)whereq(told)isthe

numberofcustomersinqueuejustbeforetheevent.responseTimeCtr +=(tnew ‐ told). q(told)

thusresponseTimeCtr == queueLengthCtr andthus

N=RxNbCust/T;nowNbCust/Tisourestimatorof

7

Other Operational Laws

8

The Interactive User Model

99

Network Laws

10

Bottleneck AnalysisApplythefollowingtwobounds1.2.

11

Example

(1)

(2)

17

Throughput Bounds

12

Bottlenecks

13

A

DASSA

Intuition:withinonebusyperiod:toeverydeparturewecanassociateonearrivalwithsamenumberofcustomersleftbehind

14

3. Single Server Queue

15

16

i.e. which are event averages (vs time averages ?)

17

18

19

2 4 6 8 10Requests per Second

0.5

1

1.5

2

2.5

Mean Response Time in seconds

Non Linearity of Response Time

20

Impact of Variability

21

0.2 0.4 0.6 0.8Utilization

2

4

6

8

10

12

14

Mean Response Time

Optimal SharingComparethetwointermsof

ResponsetimeCapacity

22

The Processor Sharing Queue

Models:processors,networklinks

Insensitivity:whatevertheservicerequirements:

Egalitarian

23

PS versus FIFO

PS FIFO

24

4. A Case Study

Impactofcapacityincrease?OptimalCapacity?

25

Methodology

26

4.1. Deterministic Analysis

27

Deterministic Analysis

28

4.2 Single Queue Analysis

29

Assumenofeedbackloop:

4.3 Operational AnalysisArefinedmodel,withcirculatingusers

ApplyBottleneckAnalysis(=OperationalAnalysis)

30

Z/(N-1)

-Z

1/c

waiting time

31

32

5. Networks of QueuesStability

QueuingnetworksarefrequentlyusedmodelsThestabilityissuemay,ingeneral,beahardone

Necessaryconditionforstability(NaturalCondition)

serverutilization<1

ateveryqueue

33

Instability Examples

34

Poissonarrivals ;jobsgothrough stations1,2,1,2,1then leaveAjobarrivesastype1,thenbecomes 2,then 3etcExponential,independentservicetimeswith meanmi

Priority schedulingStation1:5>3>1Station2:2>4

Q:What is thenaturalstability condition?A: λ (m1 +m3 + m5 )<1

λ (m2 + m4) < 1

λ =1m1 =m3 =m4 = 0.1m2 =m5 = 0.6Utilizationfactors

Station1:0.8Station2:0.7

Networkisunstable!

Ifλ (m1 +… +m5 )<1networkisstable;why?

35

Bramson’s Example 1:  A Simple FIFO Network

Poissonarrivals;jobsgothroughstationsA,B,B…,B,AthenleaveExponential,independentservicetimes

Steps2andlast:meanisLOthersteps:meanisS

Q:Whatisthenaturalstabilitycondition?A: λ (L +S )<1

λ ((J‐1)S +L )<1Bramsonshowed:maybeunstablewhereasnaturalstabilityconditionholds

Bramson’s Example 2A FIFO Network with Arbitrarily Small Utilization Factor

m queues2typesofcustomersλ =0.5eachtyperoutingasshown,…=7visitsFIFOExponentialservicetimes,withmeanasshown

37

L LS L LS S S S S S S

Utilization factorat every station≤4λ SNetworkis unstable forS ≤0.01L ≤S8m =floor(‐2(logL )/L)

Take Home Message

Thenatural stability conditionis necessary butmay notbe sufficient

Thereis aclassofnetworkswhere this never happens.ProductFormQueuingNetworks

38

Product Form Networks

Customershaveaclass attributeCustomersvisitstationsaccordingtoMarkovRouting

Externalarrivals,ifany,arePoisson

39

2StationsClass=step,J+3classes

Canyou reduce thenumberofclasses?

Chains

Customerscanswitchclass,butremaininthesamechain

40

ν

Chains may be open or closed

Openchain=withPoissonarrivals.CustomersmusteventuallyleaveClosedchain:noarrival,nodeparture;numberofcustomersisconstant

ClosednetworkhasonlyclosedchainsOpennetworkhasonlyopenchainsMixednetworkmayhaveboth

41

42

3Stations4classes1openchain1closed chain

ν

Bramson’s Example 2A FIFO Network with Arbitrarily Small Utilization Factor

43

L LS L LS S S S S S S

2StationsMany classes2openchainsNetworkis open

Visit Rates

44

45

2Stations5classes1chainNetworkis open

Visit ratesθ11 =θ13= θ15 =θ22 =θ24 = λθsc =0otherwise

46

ν

Constraints on Stations

Stationsmustbelong toarestricted catalog ofstationsSee Section8.4forfulldescriptionWe will give commonly used examplesExample 1:GlobalProcessorSharing

OneserverRateofserveris shared equally among allcustomers presentServicerequirements forcustomers ofclassc aredrawn iid from adistributionwhich depends ontheclass(andthestation)

Example 2:DelayInfinite number ofserversServicerequirements forcustomers ofclassc aredrawn iid from adistributionwhich depends ontheclass(andthestation)Noqueuing,servicetime=servicerequirement =residence time

47

Example 3:FIFOwith BserversB serversFIFOqueueingServicerequirements forcustomers ofclassc aredrawn iid from anexponentialdistribution,independent oftheclass (butmay depend onthestation)

Example ofCategory 2(MSCCCstation):MSCCCwith BserversB serversFIFOqueueing with constraintsAt most onecustomer ofeach classis allowed inserviceServicerequirements forcustomers ofclassc aredrawn iid from anexponentialdistribution,independent oftheclass (butmay depend onthestation)

Examples 1and2areinsensitive (servicetimecan be anything)Examples 3and4arenot(servicetimemustbe exponential,same forall)

48

Saywhichnetworksatisfiesthehypothesesforproductform

49

A

B (FIFO, Exp)C (Prio, Exp)

The Product Form Theorem

Ifanetworksatisfies the« ProductForm »conditionsgiven earlierThestationary distrib ofnumbers ofcustomers can be written explicitlyItis aproduct ofterms,where each term depends only onthestationEfficientalgorithms exist tocompute performancemetrics foreven very largenetworks

ForPSandDelaystations,servicetimedistributiondoes notmatter other thanthrough its mean (insensitivity)

Thenatural stability conditionholds

50

51

52

Conclusions

Queuingisessentialincommunicationandinformationsystems

M/M/1,M/GI/1,M/G/1/PSandvariantshaveclosedforms

BottleneckanalysisandworstcaseanalysisareusuallyverysimpleandoftengivegoodinsightsQueuingnetworksmaybeverycomplextoanalyzeexceptifproductform–beabletorecognizethem!

53