Upload
franklin-hancock
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
Design and Evaluation of a Model for Multi-tiered Internet Applications
Bhuvan UrgaonkarInternship project talk – Services Management Middleware
Dept, IBMAug. 20, 2004
Internet Data Centers
Internet applications run on data centers Server farms
Provide computational and storage resources
Service-level agreements Response time
guarantees Problem: Need good application models to
determine right resource allocations
Multi-tiered Applications
Internet applications: multiple tiers Example: 3 tiers: HTTP, J2EE app
server, database Replicable components
Example: clustered HTTP, J2EE server
requests
http J2EE
database Load balancing gateway
Existing Application Models
Several models for single-tier apps Queuing models for web servers: Chase et
al (USITS 03), Chandra et al (IWQoS 2003) PODC 2004: G/G/1 based model
Model only one (bottleneck) tier Ranjan et al (IWQoS 2002), Villela et al
(IWQoS 2004)
Black-box Approach Black-box approach
Treat application as a black-box Measure response time from outside Increase allocation if response time > SLA
Use a model to decide how much to allocate Strawman #1: black-box for multi-tier apps Problems:
Unclear which tier needs more capacity Bottleneck tier may not be replicable
Black-box Approach
Extension of Single-tier Model
Strawman #2: use single-tier provisioning independently at each tier
Example: Breakdown resp time into per-tier delays Use G/G/1 model for each tier
Problems: How to breakdown resp time? G/G/1 based model found to be very
conservative! Wasted capacity
Talk Outline
Motivation Multi-tier Application Model Preliminary Evaluation Ongoing Work / Discussion Summary
Key Insights/Observations
Tier i requests service from tier (i+1) Scheduling: PS closest tractable among
policies Session-based workloads:
A session consists of a succession of requests Think times are user dependent
Quantity of interest: Per-request resp time
requests
http J2EE
database Load balancing gateway
Queuing-theoretic Model Example: 3-tier application
Natural model: Network of queues shown below
. . . Capturing session-based workload
Infinite server system Closed-queuing system
X_1 X_2 X_3Z
Mean-value Analysis MVA algorithm
Inputs: Avg service times, visit ratios Computes avg. delays and resp. time
. . .
E[X_1] E[X_2] E[X_3]E[Z]
V_0=1 V_1 V_2 V_3Visitratios
MVA Algorithm
for m = 1 to M do Qm = 0for n = 1 to N do begin
for m = 1 to M doRm = Dm for the infinite server
Rm = Dm*(1+Qm) for other servers
X = n / Σ Rm ……… throughput
for m = 1 to M do Qm = X*Rm …… Little’s Lawend
Key relation: Am(N) = Qm(N-1) Am : # customers an arriving customer finds in queue m Qm : # customers in queue m
MVA Algorithm: Discussion Can handle any service time distrib
if scheduling discipline is PS Extension to multiple classes exists
Need to measure service times and visit ratios on a per-class basis
Gives only averages, not distribs Each queue is really only modeling
a single resource
Finding Model Parameters
Visit ratios Easy to obtain from various logs E.g. Apache-tomcat-mysql
V_apache = 2 V_mysql = avg # queries per servlet V_tomcat = V_mysql + 1
Finding Model Parameters Service times
Apache and Tomcat can be made to log time spent at and beyond them
X_apache (T_apache–T_tomcat)/2
X_tomat (T_tomcat–V_mysql*X_mysql)/(V_mysql+1)
X_mysql avg. query exec. time
What we haven’t captured …
Inter-tier load balancers Resources held at tier i while awaiting
response from tier (i+1) Increased service times at high loads
E.g. context switches, protocol processing, contention for locks
Tails of response times Multiple resources Load imbalances due to session affinity
Test Applications
RUBiS (eBay like auction app) BrowseItems, PutBid, AuthorizeBid, PutComment,
RegisterUser, SellItem, SearchItem, StoreBid, …
RUBBoS (slashdot like b-board app)
AcceptStory, BrowseStories, ModerateComment, PostComment, RegisterUser, RegisterStory, ViewStory, …
Experimental Setup
Rubis (e-auctions), Rubbos (b-board) Apache, Tomcat, Mysql
Apache mod_jk redirector Tiers 1 and 2 are replicable Java client
Average think time = 1 sec One thread per session
Apache+mod_jk Tomcat MysqlClient
RUBiS: Response Times
Model works well in a restricted region Tomcat had a connection limit of 75
rubis
0
5000
10000
15000
20000
25000
30000
0 100 200 300 400 500
Num sessions
Avg
resp
tim
e (m
sec)
Obs at apache
Obs at tomcat
Pred at apache
Pred at tomcat
rubis
0
1000
2000
3000
4000
5000
0 20 40 60 80 100
Num sessionsAv
g re
sp ti
me
(mse
c)
Obs at apache
Obs at tomcat
Pred at apache
Pred at tomcat
RUBiS: CPU Utilizations
rubis
0
1000
2000
3000
4000
5000
0 20 40 60 80 100
Num sessions
Avg
resp
tim
e (m
sec)
Obs at apache
Obs at tomcat
Pred at apache
Pred at tomcat
rubis - cpu utils
0
20
40
60
80
100
0 20 40 60 80 100
Num sessionsAv
g cp
u ut
il
Apache
Tomcat
Mysql
Client
App tier is the bottleneck
RUBiS: Processes and Conns.
rubis - num processes
020406080
100120140160180200
0 20 40 60 80 100
Num sessions
Num
pro
cess
es
Apache
Tomcat
Mysql
Client
rubis - num tcp conns
0
50
100
150
200
250
300
0 20 40 60 80 100
Num sessions
Num
tcp co
nns
Apache
Tomcat
Mysql
Client
RUBBoS: Response Times
Again, works well in a restricted region
rubbos
0
2000
4000
6000
8000
10000
12000
0 100 200 300 400 500
Num sessions
Avg r
esp t
ime (
mse
c)
Obs at apache
Obs at tomcat
Pred at apache
Pred at tomcat
rubbos
0
300
600
900
1200
1500
0 20 40 60 80 100
Num sessionsAv
g res
p tim
e (m
sec)
Obs at apache
Obs at tomcat
Pred at apache
Pred at tomcat
RUBiS: DB-intensive workload
Replaced SELECT with SELECT SQL_NOCACHE
rubis - resp times
0
5000
10000
15000
0 20 40 60 80 100
Num sessions
Avg r
esp t
ime (
mse
c)
Obs at apache
Obs at tomcat
Pred at apache
Pred at tomcat
Database tier is bottleneck
rubis - cpu utils
0
20
40
60
80
100
0 20 40 60 80 100
Num sessions
CPU
util
Apache
Tomcat
Mysql
Client
Query Caching at the Database
Able to capture effect of query caching at DB Interesting to do: Caching at app tier
Reduced visit ratio at database
rubis - resp times
0
5000
10000
15000
0 20 40 60 80 100
Num sessions
Avg
resp
tim
e (m
sec)
Obs at apache
Obs at tomcat
Pred at apache
Pred at tomcat
rubis - caching at database
0
3000
6000
9000
12000
15000
0 20 40 60 80 100
Num sessions
Avg
resp
tim
e (m
sec) Obs at apache
Obs at tomcat
Pred at apache
Pred at tomcat
Multiple Classes of Sessions
Class 1 : App server intensive Class 2 : Database intensive
rubis - 10 sess of Class 1
0
5000
10000
15000
0 20 40 60 80 100
Num sess of Class 2
Avg
resp
tim
e (m
sec) Obs Class 1
Obs Class 2
Pred Class 1
Pred Class 2
rubis - 10 sess pf Class 2
0
500
1000
1500
2000
0 10 20 30 40 50
Num sess of Class 1
Avg
resp
tim
e (m
sec) Obs Class 1
Obs Class 2Pred Class 1Pred Class 2
Talk Outline
Motivation Multi-tier Application Model Preliminary Evaluation Ongoing Work / Discussion Summary
Multiple Servers at a Tier
Apache+mod_jk
Tomcat
MysqlClient
Apache+mod_jk
Tomcat
MysqlClient
Load Imbalance
rubis - 2 app servers
0
500
1000
1500
2000
2500
1 2 5 10 20 50 80 100
Num sessions
Av
g r
es
p t
ime
(m
se
c)
Obs at apache
Obs at tomcat1
Obs at tomcat2
Ongoing: Introduce a skew factor for adjusting the visit ratios to the servers in the replicated tier
Session affinity Variable session
requirements
Applying the Model to more Apps/Implementations
EJB based implementations of RUBiS and RUBBoS
TPC-W Workloads that stress resources other
than CPU More??
Misc. Issues/Discussion Investigate utility of the model in
Capacity planning Dynamic provisioning Admission control
Measurements How many observations to gather?
Handling incr. svc times at higher loads Context switching, locks, protocol processing, …
Response time tails
Summary
Network of queues based model Experimental evaluation for 2 apps
Model works well in limited operating regions Simple enhancements to handle multiple
classes, multiple servers, load imbalance More work needed on several aspects
updating service times, how many observations, resp time tails, more apps and workloads
Acknowledgements
Mike Spreitzer
Asser Tantawi
Giovanni Pacifici
Thank you!
Extension of Single-tier Model