View
224
Download
2
Tags:
Embed Size (px)
Citation preview
Fundamental Characteristics of Queues with Fluctuating Load
VARUN GUPTA
Joint with:
Mor Harchol-Balter
Carnegie Mellon Univ.
Alan Scheller-Wolf
Carnegie Mellon Univ.
Uri Yechiali
Tel Aviv Univ.
2
Motivation
ClientsServer Farm
Requests
3
Motivation
ClientsServer Farm
Requests
4
Motivation
ClientsServer Farm
Requests
5
Motivation
ClientsServer Farm
Requests
6
Motivation
ClientsServer Farm
Requests
7
Motivation
ClientsServer Farm
Requests
8
Motivation
ClientsServer Farm
Requests
9
Motivation
ClientsServer Farm
RequestsReal
World Fluctuating arrival
and service intensities
10
A Simple Model
HL
exp(H)
exp(L)
HighLoad
LowLoad
11
• Poisson Arrivals• Exponential Job Size Distribution• H/H > L/L
• H>H possible, only need stability
A Simple Model
HighLoad
LowLoad
H,H
L,L
exp()
exp()
HH
LL
12
The Markov ChainP
has
e
Number of jobs
L
H
H
H
0 1
0 1
2
2
L
L
H
H
L
L
. . .
. . .
Solving the Markov chain provides no behavioral insight
13
HH
LL
• N = Number of jobs in the fluctuating load system
• Lets try approximating N using (simpler) non-fluctuating systems
14
HH
LL
Method 1
Nmix
15
HH
LL
Q: Is Nmix ≈ N?
A: Only when 0
Method 1
Nmix
½
½
+
,
16
HH
LL
Method 2
17
avg(H,L)avg(H,L)
Method 2
≡ Navg
Q: Is Navg ≈ N?
A: When ,
18
Example
H=1, H=0.99
L=1, L=0.01
E[Nmix] ≈ 49.5 E[Navg] = 1
0
19
Observations
• Fluctuating system can be worse than non-fluctuating
0 and asymptotes can be very far apart
E[Nmix] > E[Navg]
E[Nmix] E[Navg]
20
Questions
• Is fluctuation always bad?
• Is E[N] monotonic in ?
• Is there a simple closed form approximation for E[N] for intermediate ’s?
• How do queue lengths during High Load and Low Load phase compare? How do they compare with Navg?
More than 40 years of research has not
addressed such fundamental questions!
21
Outline
Is E[Nmix] ≥ E[Navg], always?
Is E[N] monotonic in ?
Simple closed form approximation for E[N]
Application: Capacity Planning
Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase Not covered in this talk
Please read paper.
22
Prior Work
Fluid/DiffusionApproximations
Transforms Matrix Analytic& Spectral Analysis
- P. Harrison- Adan and Kulkarni
Numerical ApproachesInvolves solution of cubic
- Clarke- Neuts- Yechiali and Naor
Involves solution of cubic
- Massey- Newell- Abate, Choudhary, Whitt
Limiting Behavior
But cubic equations have a close form solution…
?
23
Good luck understanding this!
24
Asymptotics for E[N] (H<H)
E[Navg]
E[Nmix]
E[N]
(switching rate)Highfluctuation
H=1, H=0.99
L=1, L=0.01
E[Nmix] > E[Navg]
Lowfluctuation
25
Asymptotics for E[N] (H<H)
E[N]
E[Nmix]
E[Navg]
• Agrees with our example (H = L)
• Ross’s conjecture for systems with constant service rate:
“Fluctuation increases mean delay”
Q: Is this behavior possible?
A: Yes
E[N]
E[Navg]
E[Nmix]
26
Our Results
E[N]
(H-H) > (L-L)(H-H) = (L-L)(H-H) < (L-L)
• Define the slacks during L and H as• sL = L - L
• sH = H - H
E[N]
E[N]
27
Our Results
• Define the slacks during L and H as• sL = L - L
• sH = H - H
• Not load but slacks determine the response times!
sH > sLsH = sLsH < sL
KEY IDEA
E[N]
E[N]
E[N]
28
Outline
Is E[Nmix] ≥ E[Navg], always?
Is E[N] monotonic in ?
Simple closed form approximation for E[N]
Application: Capacity Planning
Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase Not covered in this talk
Please read paper.
29
Outline
Is E[Nmix] ≥ E[Navg], always? No
Is E[N] monotonic in ?
Simple closed form approximation for E[N]
Application: Capacity Planning
Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase Not covered in this talk
Please read paper.
30
Notation
• NH: Number of jobs in system during H phase
• NL: Number of jobs in system during L phase
• N = (NH+NL)/2
H,H
L,L
exp()
exp()NH NL
31
Analysis of E[N]
First steps:
– Note that it suffices to look at switching points
– Express
• NL = f(NH)
• NH = g(NL)
– The problem reduces to finding Pr{NH=0} and Pr{NL=0}
H,H
L,L
NH NL
NL=f(g(NL))
fg
32
– Find the root of a cubic (the characteristic matrix polynomial in the Spectral Expansion method)
– Express E[N] in terms of
E[N] =
The simple way forward…
H,H
L,L
fg
A
A-A
H(L -L)0H+ L(H-H)0
L - (L -L)(H-H)
2 (A -A)+
Where 0L = 0
H = (A-A)
L(-1)(H-H)
(A-A)
H(-1)(L-L)
NH NL Difficult to even prove the monotonicity of E[N] wrt
using this!
33
Our approach (contd.)
• Express the first moment as
E[N] = f1()r+f0()(1-r)
– r is the root of a (different) cubic– r1 as 0 and r0 as
KEY IDEA
34
Monotonicity of E[N]
• E[N] = f1()r+f0()(1-r)
• r is monotonic in E[N] is monotonic in • The cubic for r has maximum power of as 2
1
0
r
35
Monotonicity of E[N]
• E[N] = f1()r+f0()(1-r)
• r is monotonic in E[N] is monotonic in • The cubic for r has maximum power of as 2
1
0
r
Need at least 3 roots for when r=c1
but has at most 2 roots
c1
36
Monotonicity of E[N]
• E[N] = f1()r+f0()(1-r)
• r is monotonic in E[N] is monotonic in • The cubic for r has maximum power of as 2
1
0
r
Need at least 2 positive roots for when r=c2
but for r>1 product of roots is negative
c2
37
Monotonicity of E[N]
• E[N] = f1()r+f0()(1-r)
• r is monotonic in E[N] is monotonic in • The cubic for r has maximum power of as 2
1
0
r
E[N] is monotonic in !
38
Outline
Is E[Nmix] ≥ E[Navg], always? No
Is E[N] monotonic in ?
Simple closed form approximation for E[N]
Application: Capacity Planning
Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase Not covered in this talk
Please read paper.
39
Outline
Is E[Nmix] ≥ E[Navg], always? No
Is E[N] monotonic in ? Yes
Simple closed form approximation for E[N]
Application: Capacity Planning
Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase Not covered in this talk
Please read paper.
40
Approximating E[N]
• Express the first moment as
E[N] = f1()r+f0()(1-r)
– r is the root of a (different) cubic– r1 as 0 and r0 as
• Approximate r by the root of a quadratic
KEY IDEA
KEY IDEA
41
Approximating E[N]
1
3
5
7
9
1.E-05 1.E-04 1.E-03 1.E-02 1.E-01 1.E+00 1.E+0110-5 10-4 10-3 10-2 10-1 100 101
3
5
7
9
E[N]
ExactApprox.
H=L=1, H=0.95, L=0.2
42
Approximating E[N]
1
3
5
7
9
1.E-05 1.E-04 1.E-03 1.E-02 1.E-01 1.E+00 1.E+0110-5 10-4 10-3 10-2 10-1 100 101
3
5
7
9
E[N]
ExactApprox.
H=L=1, H=0.95, L=0.2
43
Approximating E[N]
2
6
10
14
18
1.E-02 1.E-01 1.E+00 1.E+0110-2 10-1 100 10
ExactApprox.
H=L=1, H=1.2, L=0.2
2
6
10
14
18
E[N]
44
Approximating E[N]
2
6
10
14
18
1.E-02 1.E-01 1.E+00 1.E+0110-2 10-1 100 10
ExactApprox.
H=L=1, H=1.2, L=0.2
2
6
10
14
18
E[N]
45
Outline
Is E[Nmix] ≥ E[Navg], always? No
Is E[N] monotonic in ? Yes
Simple closed form approximation for E[N]
Application: Capacity Planning
Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase Not covered in this talk
Please read paper.
46
Outline
Is E[Nmix] ≥ E[Navg], always? No
Is E[N] monotonic in ? Yes
Simple closed form approximation for E[N]
Application: Capacity Planning
Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase Not covered in this talk
Please read paper.
47
Scenario
Application: Capacity Provisioning
HH
LL
2HH
2LL
Aim: To keep the mean response times same
48
Scenario
Application: Capacity Provisioning
HH
LL
2H2H
2L2L
Question: What is the effect of doubling the arrival and service rates on the mean response time?
49
What happens to the mean response time when , are doubled in the fluctuating load queue?
Halves
Remains almost the sameReduces by less than half
Reduces by more than halfA:
D:C:
B:
50
What happens to the mean response time when , are doubled in the fluctuating load queue?
Halves
Remains almost the sameReduces by less than half
Reduces by more than halfA:
D:C:
B:
51
What happens to the mean response time when , are doubled in the fluctuating load queue?
Halves
Remains almost the sameReduces by less than half
Reduces by more than halfA:
D:C:
B:
Look at slacks!
A: sH = sL
B: sH > sL
C: sH < sL
D: sH < 0, 0
reduces by half more than half less than half remains same
52
Our Contributions
• Give a simple characterization of the behavior of E[N] vs.
• Provide simple (and tight) quadratic approximations for E[N]
• Prove the first stochastic ordering results for the fluctuating load model (see paper)
53
Bon Appetit!
54
Direction for future research
• Analysis of higher moments of response time
• Analysis of bursty arrival process
• General phase type distributions for phase lengths
• Analysis of alternating traffic streams – look at the workload process instead of number of jobs in system
• Conjecture: NH increases stochastically as switching rates decrease
55
Comparison of NL vs. NH
Jackpot!
Honey, I think we chose the wrong time to go out!
56
Stochastic Ordering refresher
• Random variable X stochastically dominates (is stochastically larger than) Y if:
Pr{Xi} Pr{Yi}
for all i.
• If X stY then E[f(X)] E[f(Y)] for all increasing f– E[Xk] E[Yk] for all k 0.
57
Comparison of NL vs NH
• NL ≥st NM/M/1/L
• NH ≤st NM/M/1/H
• NH ≥st NL
• NH ≥st Navg
• NL st Navg
58
Why do slacks matter?
• Fact: The mean response time in an M/M/1 queue is (-)-1
– Higher slacks Lower mean response times
• What is the fraction of customers departing during H
H,H
L,L
exp()
exp()
when ?
H,H
L,L
exp()
exp()
when 0?
H
H + L
H
H + L?
59
Why do slacks matter?
when ?
H,H
L,L
exp()
exp()
when 0?
H
H + L
H
H + L?
• Fact: The mean response time in an M/M/1 queue is (-)-1
– Higher slacks Lower mean response times
• What is the fraction of customers departing during H
H,H
L,L
exp()
exp()
60
Why do slacks matter?
when ?
H,H
L,L
exp()
exp()
when 0?
A H?
• Fact: The mean response time in an M/M/1 queue is (-)-1
– Higher slacks Lower mean response times
• What is the fraction of customers departing during H
As switching rates decrease, larger fraction of customers experience lower mean response times when sH>sL
H,H
L,L
exp()
exp()
61
Q: What happens to E[N] when we double ’s and ’s?
A:System A: , ,
System B: 2, 2,
?
62
Q: What happens to E[N] when we double ’s and ’s?
A:System A: , ,
System B: 2, 2,
System C: 2, 2, 2
E[N] remains same in going from A to C
A) sL = sH : remains same
B) sL > sH : increases, but by less than twice
C) sL < sH : decreases
D) 0, H>1 : queue lengths become twice as switching rates halve, E[N] doubles
63
Example
H=1.9, H=0.99
L=0.1, L=0.01
E[Nmix] ≈ 0.6 E[Navg] = 1
0