Upload
jordanfishman
View
220
Download
0
Embed Size (px)
Citation preview
8/3/2019 Performance U Can Use
1/53
1
Performance Theory
You can useUF Platform and Product Development
Robert Vanderwall
Date
8/3/2019 Performance U Can Use
2/53
2 Code Camp 2012
It is necessary to combine knowledge born from study with
sincere practice in our daily lives. These two must go together.
-- Dalai Lama
"Theory without practice is empty; Practice without theory is
blind."
-- Immanual Kant
Theory + practice = Expertise
8/3/2019 Performance U Can Use
3/53
3 Code Camp 2012
Topics
Performance Engineering
Arrivals
Throughput
Queuing models
Practical Implications
8/3/2019 Performance U Can Use
4/53
4 Code Camp 2012
Topics
Performance Engineering
Arrivals
Throughput
Queuing models
Practical Implications
8/3/2019 Performance U Can Use
5/53
5 Code Camp 2012
Performance Engineering
Establish a target
Response time < 1.0 seconds for 90% of users under 1000 user load.
Model the system
Single Incoming queue, N servers.
Budget the time to each component
10 mSec network traversal, 50 mSec data query, etc.
Estimate response using known parameters, queuing theory.
Measure the actual system
Adjust
8/3/2019 Performance U Can Use
6/53
6 Code Camp 2012
Topics
Performance Engineering
Arrivals
Throughput
Queuing models
Practical Implications
8/3/2019 Performance U Can Use
7/53
7 Code Camp 2012
Arrivals
MEASURE MEANING
P Arrival Rate Arrivals per second.
XS Service Time Time to actually process arequest.
8/3/2019 Performance U Can Use
8/53
8 Code Camp 2012
Arrivals Uniform arrivals
XS
1/P
Each 1/P seconds, exactly 1 request arrives. Eachrequest takes exactly XS seconds to complete.
Example: P = 1/sec. and XS = 0.5 seconds.
8/3/2019 Performance U Can Use
9/53
9 Code Camp 2012
Uniform Arrivals
XS
1/P
Increase Arrival rate from P= 1/sec. to P= 1.5/sec.
Period goes from 1 sec. to 0.667 sec.
8/3/2019 Performance U Can Use
10/53
10 Code Camp 2012
Uniform Arrivals
Continue to increase Arrival rate to P= 2.0/sec.
Period goes from 1 sec. to 0.5 sec.
This is the absolute best case
Pnom = XSXS
1/Pnom
8/3/2019 Performance U Can Use
11/53
11 Code Camp 2012
Uniform Arrival
Increase Arrival rate beyond P= 2.0/sec. andrequests overlap.
If there is no queue, then requests are lost
Serviced
Dropped
Serviced
8/3/2019 Performance U Can Use
12/53
12 Code Camp 2012
Uniform Arrival Past nominal with queue
Add a queue, and requests get queued instead oflost.
Queued
Queued
8/3/2019 Performance U Can Use
13/53
13 Code Camp 2012
Uniform Arrival Past nominal with queue
Queue will continue to grow without bounds
1 2 3 4 5 6 7 8 161514131211109
1 2 6543 10987
0 1 1 1 2 2 2 2 2 3 average queue size
8/3/2019 Performance U Can Use
14/53
14 Code Camp 2012
Non-Uniform arrivals
Fortunately, in most real conditions, the arrivals are
not uniform. That is, there are bursts of activities
and lulls in activities.
During the bursts, the queues fill and during the lulls,
the queues are emptied.
8/3/2019 Performance U Can Use
15/53
15 Code Camp 2012
Typical Arrival Distribution
Arrivals often follow a Poisson distribution, not a
uniform distribution.
P(N) = e
-PPN
/N!
The expectednumber of arrivals in a time period T is
PT.
8/3/2019 Performance U Can Use
16/53
16 Code Camp 2012
Poisson distribution
Below is a plot for P= 3.
For a given interval, there is about a 5% probability that no
requests arrive
about a 15% chance that 1 will arrive,
about 22% chance that 2 will arrive
about 22% chance for 3.
There is an almost 0% chance that 9 or 10 will arrive.
0.224042
8.101512104
p n( )
100 n
0 2 4 6 8 100
0.1
0.2
0.3
0.224042
8.101512104
p n( )
100 n
0 2 4 6 8 100
0.1
0.2
0.3
8/3/2019 Performance U Can Use
17/53
17 Code Camp 2012
Takeaways
Defined Arrival rates and service time.
If arrivals are too quick:
Drop requests
Utilization may actually reduce Queues fill and overflow
Arrivals are typically not uniform.
8/3/2019 Performance U Can Use
18/53
18 Code Camp 2012
Topics
Performance Engineering
Arrivals
Throughput
Queuing models
Practical Implications
8/3/2019 Performance U Can Use
19/53
19 Code Camp 2012
Canonical Throughput Graph
Suppose a system can process a request within S seconds(service time) and N requests arrive each second (arrival
rate).
When S < 1/N, the arrival rate is low enough that the system
can process each request as it arrives. In a one second period, N requests will arrive and they will all be
serviced within that time. This yields a throughput of N requests per
second. As N increases, the throughput is equal to the arrival rate
(within a multiplicative constant for units), or T = N.
This is depicted in the linear portion of the graph.
8/3/2019 Performance U Can Use
20/53
20 Code Camp 2012
Canonical Throughput Graph
User Request (N)
Throughput
T=1/S
T = N region
8/3/2019 Performance U Can Use
21/53
21 Code Camp 2012
Canonical Throughput Graph
If the arrival rate exceeds 1/S, then the system has saturated
and the throughput is now constant regardless of increases in
N.
8/3/2019 Performance U Can Use
22/53
22 Code Camp 2012
Canonical Throughput Graph
User Request (N)
Throughput
T=1/S
T = constant region
8/3/2019 Performance U Can Use
23/53
23 Code Camp 2012
Canonical Throughput Graph
User Request (N)
Throughput
Ideal Throughput graph
8/3/2019 Performance U Can Use
24/53
24 Code Camp 2012
Realistic Throughput
Several things prevent a real curve from looking like the ideal
curve.
First, the actual peak throughput will not quite meet the ideal
expectations due to many different effects.
Second, as the number of requests increases, other effects
come into play as well.
Things such as thrashing, resource contention, and bottlenecks also
reduce the realized throughput as N increases beyond the saturation
point. The end result is the elephant graph as shown in the nextgraph.
8/3/2019 Performance U Can Use
25/53
25 Code Camp 2012
More realistic graph
User Request (N)
Throughput Knee of the graph
8/3/2019 Performance U Can Use
26/53
26 Code Camp 2012
Other measures
Client Bandwidth, MB/s
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70
Client Response Time, s
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
0 10 20 30 40 50 60 70
While these are common, and somewhat useful, they dont give the insight
that the aggregate bandwidth graph does.
8/3/2019 Performance U Can Use
27/53
8/3/2019 Performance U Can Use
28/53
28 Code Camp 2012
Partial graph
User Request (N)
Throughput
We did not push the system into saturation. i.e.,
didnt find the knee.
We need to test with larger values of N.
What can we say about this graph?
8/3/2019 Performance U Can Use
29/53
29 Code Camp 2012
Takeaways
Plot the aggregate throughput. This gives you the most
insight into how the system behaves.
If you dont see a clear knee, continue to increase N.
Aggregate throughput = total number of transactions/sec.
8/3/2019 Performance U Can Use
30/53
30 Code Camp 2012
Topics
Performance Engineering
Arrivals
Throughput
Queuing models
Practical Implications
8/3/2019 Performance U Can Use
31/53
31 Code Camp 2012
Queuing Theory - Vocabulary
MEASURE MEANING
U Utilization
P Arrival Rate
XS Service Time Time to actually process arequest
Xr Resident Time Total time in system
XR Response Time
Xt
Think Time
N Requests in the system (queued)
8/3/2019 Performance U Can Use
32/53
32 Code Camp 2012
Three (or 4) Identities
N =Ptr (Littles Law) U = P ts (Utilization)
tr = ts/(1-U) (Response)
Useful rearrangement:
ts = tr/(1+ P tr)
Tr U( )
U
0 10 20 30 40 50 60 70 80 90 1000
2
4
6
8
10
12
14
16
18
20
8/3/2019 Performance U Can Use
33/53
8/3/2019 Performance U Can Use
34/53
34 Code Camp 2012
Littles Law: N =Ptr
Example: A restaurant can seat 100 people. People take
about 1 hour for dinner.
tr = 1 hour, N = 100 people.P = N/ tr = 100p/h.
If there are 15 people in queue ahead of you, how long is the
wait? P = 1.67 p/minute = 1P / 37 seconds. T = N/ P = 9.25 minutes
8/3/2019 Performance U Can Use
35/53
8/3/2019 Performance U Can Use
36/53
36 Code Camp 2012
Utilization
Suppose it takes 500 mSec (0.5 sec) to service a request and
we get 1 request per second. What utilization will we see?
U = P ts = 1 * 0.5 = 50%
Suppose we get 3 requests per second?
U = P ts = 3 * 0.5 = 150%
What does this mean?
Weve got more than one server (CPU core) OR
We are dropping requests.
8/3/2019 Performance U Can Use
37/53
37 Code Camp 2012
Response Time Function
tr = ts/(1-U)
Tr U( )
U
0 10 20 30 40 50 60 70 80 90 1000
2
4
6
8
10
12
14
16
18
20
8/3/2019 Performance U Can Use
38/53
38 Code Camp 2012
Response time
Suppose we have a load of50 requests per second and the
service time is 15 mSec. What is the expected response time?
U = P ts = 50 * 0.015 = 0.75 (75%)
tr = ts/(1-U) = 0.015/(1 0.75) = 0.060 = 60 mSec.
What if we add a second server? U = 37.5%
tr = ts/(1-U) =0.0
15
/(1 0.375
) =0.0
24
= 24
mSec.
8/3/2019 Performance U Can Use
39/53
39 Code Camp 2012
Takeaways
With only three equations, you can do a lot of analysis:
N =Ptr (Littles Law)
U = P ts (Utilization)
tr = ts/(1-U) (Response)
8/3/2019 Performance U Can Use
40/53
40 Code Camp 2012
Topics
Performance Engineering
Arrivals and Littles Law
Throughput
Queuing models Practical Implications
8/3/2019 Performance U Can Use
41/53
41 Code Camp 2012
Simple Client-server example
Client Server
Network Latency - XLRequest
Processing time - XS
Response Time - Xr
Xr= XL + XS + XL + XTNOTE: This is only a model, and hence it is wrong, but still useful.
Transport Time - XT
Network Latency - XLResponse
8/3/2019 Performance U Can Use
42/53
42 Code Camp 2012
Not so Simple case
Single processor, wide transport pipe.
Client Server
Request
Processing time - XS
Total Response
Time -X
tr
Xtr= XL + XS + XS + XL + XT
Processing time - XS
8/3/2019 Performance U Can Use
43/53
43 Code Camp 2012
Not so Simple case
Two processor, wide transport pipe.
Client Server
Request
Processing time - XS
Total Response
Time -X
tr
Xtr= XL + XS + XL + XT
8/3/2019 Performance U Can Use
44/53
44 Code Camp 2012
Multi-server
Suppose we have 4 cores.
And, 1 request takes 100 mSec.
How long will it take for 2 requests?
for 4 requests?
for 5 requests?
for 8 requests?
8/3/2019 Performance U Can Use
45/53
45 Code Camp 2012
Multiple cores
We expect that multiple cores will multiply the throughput by
the number of cores.
If we can get 200 transactions per second with one core, we
expect roughly 800 transactions per second with 4 cores.
If we dont get that, what do we need to look for?
Another bottleneck!!
8/3/2019 Performance U Can Use
46/53
46 Code Camp 2012
Bottlenecks
There are essentially only 4 bottlenecks:
CPU
Memory
Disk Network
8/3/2019 Performance U Can Use
47/53
47 Code Camp 2012
Takeaways
Latency often dominates when there are few arrivals or theyare serialized.
CPU tends to dominate when there are many arrivals in
parallel.
To a point, when other resources become bottlenecks.
Memory
Disk
Network
8/3/2019 Performance U Can Use
48/53
48 Code Camp 2012
Recap
8/3/2019 Performance U Can Use
49/53
49 Code Camp 2012
Takeaways
Defined Arrival rates and service time.
If arrivals are too quick:
Drop requests
Utilization may actually reduce
Queues fill and overflow
Arrivals are typically not uniform.
8/3/2019 Performance U Can Use
50/53
50 Code Camp 2012
Takeaways
Plot the aggregate throughput. This gives you the most
insight into how the system behaves.
If you dont see a clear knee, continue to increase N.
8/3/2019 Performance U Can Use
51/53
51 Code Camp 2012
Takeaways
With only three equations, you can do a lot of analysis:
N =Ptr (Littles Law)
U = P ts (Utilization)
tr = ts/(1-U) (Response)
8/3/2019 Performance U Can Use
52/53
52 Code Camp 2012
Takeaways
Latency often dominates when there are few arrivals or theyare serialized.
CPU tends to dominate when there are many arrivals in
parallel.
To a point, when other resources become bottlenecks.
Memory
Disk
Network
8/3/2019 Performance U Can Use
53/53
53 Code Camp 2012