Dynamic Provisioning for Multi-tier Internet Applications

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science

Dynamic Provisioning for Multi-tier Internet Applications

Bhuvan Urgaonkar, Prashant Shenoy, Abhishek Chandra, Pawan Goyal

University of MassachusettsUniversity of Minnesota

Veritas Software India Pvt. Ltd.

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 2

Internet Applications

Proliferation of Internet applications

auction site online game online store

Growing significance in personal, business affairs Focus: Internet server applications


Multi-tiered Internet Applications

Internet applications: multiple tiers Example: 3 tiers: HTTP, J2EE app server, database

Replicable components Individual tiers: partially or fully replicable Example: clustered HTTP, J2EE server, shared-nothing db

Each tier uses a dispatcher: load balancing

requests

http J2EE

database Load balancer


Internet Workloads Are Dynamic

Multi-time-scale variations Time-of-day, hour-of-day

Flash crowds

Key issue: How to provide desired response time under varying workloads?

0

20000

40000

60000

80000

100000

120000

140000

0 5 10 15 20

Time (hrs)

Req

ues

t R

ate

(req

/min

)

0 12 24 Time (hours)

Time (days)0 1 2 3 4 5

Arr

ival

s p

er m

in

0

0

140K

1200


Internet Data Center

Internet applications run on

data centers Server farms

o Provide computational and

storage resources

Applications share data

center resources

Problem: How should the platform allocate resources to absorb workload variations?


Our Provisioning Approach

Flexible queuing theoretic model Captures all tiers in the application

Predictive provisioning Long-term workload variations

Reactive provisioning Short-term variations, flash crowds


Talk Outline

Introduction

Internet data center model Existing provisioning approaches Dynamic capacity provisioning Implementation and evaluation Summary


Data Center Model

Dedicated hosting: each application runs on a subset of servers in the data center Subsets are mutually exclusive: no server sharing Data center hosts multiple applications

Free server pool: unused servers

Retail Web site streaming


Single-tier Provisioning

Single tier provisioning well studied [Muse] Non-trivial to extend to multiple-tiers

Strawman #1: use single-tier provisioning independently at each tier

Problem: independent tier provisioning may not increase goodput

C=15 C=10 C=10.1

14 req/s14 10 10

dropped 4 req/s


Single-tier Provisioning

Single tier provisioning well studied [Muse] Non-trivial to extend to multiple-tiers

Strawman #1: use single-tier provisioning independently at each tier

Problem: independent tier provisioning may not increase goodput

C=15 C=10.1

14 req/s14

C=20

14

dropped 3.9 req/s

10.1


Model-based Provisioning

Black box approach Treat application as a black box Measure response time from outside Increase allocation if response time > SLA

o Use a model to determine how much to allocate Strawman #2: use black box for multi-tier apps Problems:

Unclear which tier needs more capacity May not increase goodput if bottleneck tier is not replicable

14 req/s

C=15 C=10.1

14

C=20

14 10.1


Provisioning Multi-tier Apps

Approach: holistic view of multi-tier application Determine tier-specific capacity independently Allocate capacity by looking at all tiers (and other apps)

Predictive provisioning Long-term provisioning: time scale of hours Maintain long-term workload statistics Predict and provision for the next few hours

Reactive provisioning Short term provisioning: time scale of several minutes React to “current” workload trends Correct errors of long-term provisioning Handle flash crowds (inherently unpredictable)


Predictive Provisioning

Workload predictor Predicts workload based on past observations

Application model Infers capacity needed to handle given workload

Predictor Model

pastworkload

predictedworkload

requiredcapacity

response time target


Workload Prediction

Long term workload monitoring and prediction Monitor workload for multiple

days Maintain a histogram for each

hour of the dayo Capture time of day effects

Forecast based ono Observed workload for that hour

in the pasto Observed workload for the past

few hours of the current day Predict a high percentile of

expected workload

Mon

Tue

Wed

Today


Model-based Capacity Inference

Queuing theoretic application model Each individual server is a G/G/1 queue

Derive per-tier E(r) from end-to-end SLA Monitor other parameters and determine per-server

capacity) Use predicted workload pred to determine # servers per tier

o Assumes perfect load balancing in each tier

G/G/1

G/G/1

G/G/1

pred

E (s) a

2 b2

2 * E (r) E(s)

1


Reactive Provisioning

Idea: react to current conditions Useful for capturing significant short-term fluctuations Can correct errors in predictions

Track error between long-term predictions and actual Allocate additional servers if error exceeds a threshold Account for prediction errors

Can be invoked if request drop rate exceeds a threshold Handles sudden flash crowds

Operates over time scale of a few minutes Pure reactive provisioning: lags workload

Reactive + predictive more effective!

Predictionerrorpred

actual

error > Invokereactor

time series

allocate servers


Talk Outline

Introduction Internet data center model Existing provisioning approaches Dynamic capacity provisioning

Implementation and evaluation Summary


Prototype Data Center

40+ Linux servers Gigabit switches Multi-tier applications

Auction (RUBiS) Bulletin-board (RUBBoS) Apache, Tomcat (replicable) Mysql database

Control Plane Control Plane Dynamic provisioning

Nuc

leus

Apps

OS

Server Node

Applications

Resource monitoringParameter estimationN

ucle

us

Apps

OS Nuc

leus

Apps

OS


Only Predictive Provisioning

20

40

60

80

100

120

140

160

0 10 20 30 40 50 60

Arr

iva

ls p

er

min

Time (min)

Workload Response time

Auction application RUBiS Factor of 4 increase in 30 min

Predictor fails during [15, 30] resulting in under-provisioning Response time violations occur

0

1000

2000

3000

4000

5000

6000

7000

0 10 20 30 40 50 60

Time (min)

Res

p tim

e (m

sec)


Only Reactive Provisioning

20

40

60

80

100

120

140

160

0 10 20 30 40 50 60

Arr

ival

s p

er m

in

Time (min)

Workload Response time


Response time shows oscillatory behavior Several response time violations occur

0

1000

2000

3000

4000

5000

6000

7000

0 10 20 30 40 50 60Time (min)

Res

p t

ime

(mse

c)


Predictive + Reactive Provisioning

Workload Response timeServer allocations


0

2

4

6

8

10

12

0 10 20 30 40 50 60

Web serversApp servers

Nu

mbe

r of

ser

vers

Time (min)

Server allocations increased to match increased workload Response time kept below 2 seconds

20

40

60

80

100

120

140

160

0 10 20 30 40 50 60

Arr

ival

s p

er m

in

Time (min)

0

1000

2000

3000

4000

5000

6000

7000

0 10 20 30 40 50 60

Res

p t

ime

(mse

c)

Time (min)


Summary

Dynamic provisioning for multi-tier applications Flexible queuing theoretic model

o Captures all tiers in the application Predictive provisioning Reactive provisioning

Implementation and evaluation on a Linux cluster


Thank you!

More information at: http://www.cs.umass.edu/~bhuvan

Documents

Dynamic Provisioning for Multi-tier Internet Applications