UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science
An Analytical Model for Multi-tier Internet Services and its Applications
Bhuvan Urgaonkar, Giovanni Pacifici, Prashant Shenoy, Mike Spreitzer,
Asser Tantawi
University of Massachusetts and IBM TJ Watson
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 2
Internet Applications
Proliferation of Internet applications
auction site online game online store
Growing significance in personal, business affairs Focus: Modeling Internet applications
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 3
Why Model Internet Applications?
Capacity provisioning How many servers does the application need?
Performance prediction E.g., predict response time
Application configuration Tune various application parameters
Request policing Turn away excess requests during overloads
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 4
Internet Application Architecture
Multi-tier architecture Each tier uses services provided by its successor
Session-based workloads Caching, replication
HTTP J2EE Databaserequest processing in an online bookstore
search “moby” queries
response
Melville’s ‘Moby Dick’Music CDs by Moby
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 5
Existing Application Models Models for Web servers [Chandra03, Doyle03]
Do not model Java server, database etc.
Black-box models [Kamra04, Ranjan02] Unaware of bottleneck tier
Extensions of single-tier models [Welsh03] Fail to capture interactions between tiers
Existing models inadequate for multi-tier Internet applications
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 6
Talk Outline
MotivationApplication Model
Evaluation of the Model Dynamic Capacity Provisioning Summary and Future Research
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 7
Baseline Application Model
Model consists of two components Sub-system to capture behavior of clients Sub-system to capture request processing inside the application
clients application
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 8
Modeling Clients
Clients think between successive requests Infinite server system to capture think time Z Captures independence of Z from processing in application
Client 1
Client 2
Client N
Z
Z
Z
Q0
applicationclients
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 9
Modeling Request Processing
Q1 Q2 QM
tier 1 tier 2 tier M
pM=1p3p1
p2
S1 S2 SM
Transitions defined to capture circulation of requests Request may move to next queue or previous queue
Multiple requests are processed concurrently at tiers Processor sharing scheduling discipline
Caching effects get captured implicitly!
N
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 10
Putting It All Together
Q0
Q1 Q2 QM
pM=1p3p1
p2
Z
Z
S1 S2 SM
N
A closed-queuing model that captures a given number of simultaneous sessions being served
tier 1 tier 2 tier M
client
client
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 11
Model Solution and Parameter Estimation
Mean Value Analysis (MVA) Algorithm Computes mean response time
Visit ratios Equivalent to trans. probs. for MVA Vi ≈ λi / λreq ; λreq at policer, λi from logs
Service times Use residence time Xi logged at tier i For last tier, SM ≈ XM
Si = Xi – ( Vi+1 / Vi ) · Xi+1
Think time Measured at the entry point of application
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 12
Talk Outline
Motivation Application Model
Evaluation of the Model Dynamic Capacity Provisioning Summary and Future Research
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 13
Evaluation of Baseline Model
Auction site RUBiS One server per tier
Apache JBOSS Mysql
Concurrency limits not captured
0
5000
10000
15000
20000
25000
30000
0 100 200 300 400 500
ObservedBasic Model
Avg
resp
tim
e (m
sec)
Num sessions
150 75
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 14
Q0Q1 Q2 QM
Z
ZS1 S2 SM
N
Requests may be dropped due to concurrency limits Need to model the finiteness of queues!
Handling Concurrency Limits
dropped requests
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 15
QMp1 pM
S1 SM
Q0Q1 Q2 QM
Z
ZS1 S2 SM
N
Approach: Subsystems to capture dropped requests Distinguish the processing of dropped requests
Handling Concurrency Limits
dropQ1drop drop
dropdrop
drop
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 16
Enhanced model can capture concurrency limits
Response Time Prediction
0
5000
10000
15000
20000
25000
30000
0 100 200 300 400 500
ObservedBasic ModelEnh Model
Avg
resp
tim
e (m
sec)
Num sessions
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 17
Query Caching at the Database
0200400600800
100012001400
0 20 40 60 80 100
ObservedModel
Avg
resp
tim
e (m
sec)
% queries cached
Caching effects Captured by tuning Vi and/or Si
Bulletin-board site RUBBoS 50 sessions
SELECT SQL_NO_CACHE causes Mysql to not cache the response to a query
More model enhancements Replication at tiers Multiple session classes
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 18
Prototype Data Center
40+ Linux servers Gigabit switches Multi-tier applications
Auction (RUBiS) Bulletin-board (RUBBoS) Apache, JBOSS
(replicable) Mysql database
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 19
Dynamic Capacity Provisioning
01000200030004000500060007000
0 10 20 30 40 50 60
Res
p tim
e (m
sec)
Time (min)
Workload Response timeServer allocations
Auction application RUBiS Factor of 4 increase in 30 min
0
2
4
6
8
10
12
0 10 20 30 40 50 60
Web serversApp servers
Num
ber o
f ser
vers
Time (min)
20406080
100120140160
0 10 20 30 40 50 60
Arr
ival
s pe
r min
Time (min)
Server allocations increased to match increased workload Response time kept below 2 seconds
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 20
Talk Outline
Motivation Baseline Application Model Evaluation of the Model Dynamic Capacity Provisioning
Summary and Future Research
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 21
Summary and Future Work
Analytical model for Multi-tier Internet Applications Mean-value analysis Concurrency limits, replication, caching, multiple classes Model validation using 3-tier applications Dynamic provisioning, request policing
Future work Handling load imbalances at replicated tiers Handling more diverse workloads Handling other kinds of scheduling disciplines at servers
UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 22
Thank you!
More information at: http://www.cs.umass.edu/~bhuvan