Little’s Law & Operational Laws. Little’s Law Proportionality relation between the average number of jobs (E[N]) in a system and the average system time

Little’s Law & Operational Laws

2

Little’s Law• Proportionality relation between the average number of jobs (E[N]) in a

system and the average system time (E[T]) of those jobs (the job arrival rate is the proportionality constant)– No assumptions about the arrival or service processes

• Two similar versions: Open and closed systems• Little’s Law for an ergodic open system: E[N] = λE[T]

– λ is the average job arrival rate in the system

• Little’s law for an ergodic closed system: N = XE[T]– N is constant and equal to the multi-programming level– X is the system throughput (rate of job completion)– E[T] = E[R] + E[Z], where R is response time, and Z think time

– Alternative version (Response Time Law): E[R] = N/X – E[Z]

3

Applying Little’s Law – (1)

• Assume that the above system is ergodic with X = = 1+2 , where 1

(2) is he arrival rate of red (green) jobs

– Ergodicity requires <

• Utilization Law: system = server– Let (i) be fraction of time server is busy (with type i jobs) – Note: This is

also the average number of (type i) jobs in service

– Little’s Law implies = / (i = i/), or = E[S]

• Note: If E[S1] E[S2], then i = i/i, where i = E[Si]

• Time in queue: system = queue– E[NQ

i] = i E[TQi]

1

2

4

Applying Little’s Law – (2)

• Assume N = 10, E[Z] = 5secs, and E[R] = 15secs• What is the system throughput X?

N = X E[T] = X(E[Z] + E[R]) X = N/(E[R] + E[Z]) = 10/(5+15) = 0.5 jobs/sec

…

inout

N

5

Forced Flow Law

• Relates system throughput to throughput of individual device– X is system throughput, Xi is throughput of device i

– Vi is number of visit to device i per job

• Forced Flow Law: Xi = E[Vi]X– Basically accounts for the fact that for every system

completion, there must have been E[Vi] completions (visits) at device i, so the rate of completion at device i must be higher in that same proportion

6

Bottleneck Law

• Di is the per job service demand at device i, i.e., total demand across all visits to device i

– Di = Si,1 + Si,2 + … + Si,Vi

– This implies E[Di] = E[Vi]E[Si], if Vi and Si are independent (a typical scenario)

– Note that E[Di] = Bi/C, where Bi is the busy time at device i and C is the number

of service completion during that time period

• Bottleneck Law: i = XE[Di]

(i = XiE[Si] = XE[Vi]E[Si] = X E[Di])

– Rate of outside arrivals is X and each such arrival contributes E[Di] worth of

service time at device i, so that device i a fraction of time (i )equal to X E[Di]

7

Summary• Open systems– E[N] = λE[T]– E[Nsubsystem] = subsytem E[Tsubsystem]

• Closed systems– N = XE[T] = X(E[R] + E[Z])– E[R] = N/X – E[Z] (Response Time Law)– i = i/i = i E[Si] = Xi E[Si] (Utilization Law)

– Xi = E[Vi]X (Forced Flow Law)

– i = XE[Di] (Bottleneck Law)

8

Building on Little’s Law

• Bounds on throughput and response time for closed systems

where Dmax = maxi{E[Di]} (system bottleneck)

First expression holds for N small, while second expression holds for N large

• N* as the N value for which the two expressions are equal, i.e., N*

= (D+E[Z])/Dmax

– When N > N*, throughput is dominated by bottleneck component

ZEDNDRE

DZEDNX

max

max,max

1,min

9

Example (1)

• Which system has higher throughput?– First compute N* (N* = (D+E[Z])/Dmax) for both systems

• DA = DCPU + Ddisk = 8.6, DA,max = 4.6 NA* = (8.6+5)/4.6 = 2.95

• DB = DCPU + Ddisk = 6.8, DB,max = 4.9 NB* = (6.8+5)/4.9 = 2.41

• NA* & NB

* << N

– So in both cases, the throughput is dominated by Dmax, and System A wins because

it has a lower Dmax value

…Disk

N = 10, E[Z] =5

CPU

• System A: DCPU = 4.6 and Ddisk = 4.0

• System B: DCPU = 4.9 and Ddisk = 1.9

10

Example (2)• Measurements for interactive system with N = 20 and E[Z] = 15 secs:

– T = 650 secs (duration of measurements)– BCPU = 400 secs– Bslowdisk = 100 secs– Bfastdisk = 600 secs– C = CCPU = 200 jobs– Cslowdisk = 2,000 jobs– Cfastdisk = 20,000 jobs

• Improvements under consideration (are they worth it?)– Faster CPU– Rebalancing disks– Add a second fast disk, and split the load of the original fast disk using it (Note: This is

sub-optimal – should rebalance across all three disks)– Combine all three improvements, including rebalancing all three disks

• Summary– N = XE[T] = X(E[R] + E[Z])– E[R] = N/X – E[Z] (Response Time Law)– i = i/i = i E[Si] = Xi E[Si] (Utilization

Law)– Xi = E[Vi]X (Forced Flow Law)


11

Example (2a)• Measurements for interactive system with N = 20 and E[Z] = 15 secs:

– T = 650 secs (duration of measurements)– BCPU = 400 secs, Bslowdisk = 100 secs, Bfastdisk = 600 secs– C = CCPU = 200 jobs. Cslowdisk = 2,000 jobs, Cfastdisk = 20,000 jobs

• Intermediate quantities– E[DCPU] = BCPU/C = 400 secs/200 jobs = 2.0 secs/job– E[Dslowdisk] = Bslowdisk/C = 100 secs/200 jobs = 0.5 secs/job– E[Dfastdisk] = Bfastdisk/C = 600 secs/200 jobs = 3.0 secs/job– E[VCPU] = CCPU/C = 200 visits/200 jobs = 1.0 visit/job– E[Vslowdisk] = Cslowdisk/C = 2,000 visits/200 jobs = 10 visits/job– E[Vfastdisk] = Cfastdisk/C = 20,000 visits/200 jobs = 100 visits/job– E[SCPU] = BCPU/CCPU = 400 secs/200 visits = 2.0 secs/visit– E[Sslowdisk] = Bslowdisk/Cslowdisk = 100 secs/2,000 visits = 0.05 sec/visit– E[Sfastdisk] = Bfastdisk/Cfastdisk = 600 secs/20,000 visits =0.03 sec/visit

• Original system: Dmax= 3 secs/job, D = 5.5 secs/job, N* = 20.5/3 7 << N = 20– So X 0.33 jobs/sec and E[R] = 45 secs




12

Example (2) – Faster CPU• Metrics for interactive system with N = 20 and E[Z] = 15 secs:

– E[DCPU] = BCPU/C = 400 secs/200 jobs = 2.0 secs/job

– E[Dslowdisk] = Bslowdisk/C = 100 secs/200 jobs = 0.5 secs/job

– E[Dfastdisk] = Bfastdisk/C = 600 secs/200 jobs = 3.0 secs/job (bottleneck)

– E[VCPU] = CCPU/C = 200 visits/200 jobs = 1.0 visit/job

– E[Vslowdisk] = Cslowdisk/C = 2,000 visits/200 jobs = 10 visits/job

– E[Vfastdisk] = Cfastdisk/C = 20,000 visits/200 jobs = 100 visits/job

– E[SCPU] = BCPU/CCPU = 400 secs/200 visits = 2.0 secs/visit

– E[Sslowdisk] = Bslowdisk/Cslowdisk = 100 secs/2,000 visits = 0.05 sec/visit

– E[Sfastdisk] = Bfastdisk/Cfastdisk = 600 secs/20,000 visits =0.03 sec/visit

• Faster CPU (twice as fast): E[DCPU] = 1.0 sec/job, but Dmax remains unchanged and N* stays approximately constant– Hardly any improvement as the fast disk is the bottleneck




13

Example (2) – Rebalance Disks

• Metrics for interactive system with N = 20 and E[Z] = 15 secs:– E[DCPU] = BCPU/C = 400 secs/200 jobs = 2.0 secs/job


– E[Dfastdisk] = Bfastdisk/C = 600 secs/200 jobs = 3.0 secs/job







• Clearly, we have overloaded the fast disk. Optimal balancing is such that– E[Vslowdisk] E[Sslowdisk] = E[Vfastdisk] E[Sfastdisk] or E[Vslowdisk] 0.05 = E[Vfastdisk] 0.03

while keeping E[Vslowdisk] + E[Vfastdisk] = 110

– This gives E[Vfastdisk] 69 and E[Vslowdisk] 41 and consequently E[Dslowdisk] = E[Dfastdisk] = 2.06 secs, and therefore a new

value of Dmax = 2.06 secs

– D is now 6.12 secs and N* becomes 10, which remains smaller than N. Hence, throughput and response time are still dominated by Dmax, and the system improves to X 0.485 jobs/sec and E[R] = 26.2 secs




14

Example (2) – Add Another Fast Disk










• If we split the fast disk load across two fast disks, we get– E[Dfastdisk1] = E[Dfastdisk2] = 1.5 jobs/sec

– The slow disk becomes the bottleneck which yieds a new value of Dmax = 2.0 secs

– D does not change but since Dmax does, N* becomes 10. This remains smaller than N, so that the system improves to X = 0.5 jobs/sec and E[R] = 25 secs




15

Example (2) – Combine All Improvements










• With a faster CPU and two fast disks and rebalancing load across disks, we get– E[DCPU] = 1 sec/job, E[Dfastdisk1] = E[Dfastdisk2] = E[Dslowdisk] = 1.27 jobs/sec D = 4.8 secs

• Where we have used E[Vslowdisk] 0.05 = E[Vfastdisk] 0.03, and E[Vslowdisk] + 2E[Vfastdisk] = 110

– This gives Dmax = 1.27 secs and N* 16 that remains smaller than N

– Hence, throughput and response time are still dominated by Dmax, and the system improves to X 0.787 jobs/sec and E[R] = 10.4 secs




Documents

Little’s Law & Operational Laws. Little’s Law Proportionality relation between the average number of jobs (E[N]) in a system and the average system time