UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science

Dynamic Resource Allocation for Shared Data Centers Using

Online Measurements

Abhishek Chandra

Weibo Gong

Prashant Shenoy

UMASS Amherst

http://lass.cs.umass.edu/projects/shop

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 2

Motivation

Data Centers Server farms Rent computing and

storage resources to applications

Revenue for meeting QoS guarantees

Goals: Satisfy application QoS guarantees Maximize resource utilization of platform Robustness against “Slashdot” effects

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 3

Dynamic Resource Allocation

Periodically re-allocate resources among applications Estimate resource requirements for near future

Challenges: Reallocation at short time-scales No prior workload profiling/knowledge Low overhead

Approach: Online Measurement-based Allocation

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 4

Talk Outline

Motivation

System Model

Dynamic Allocation Techniques

Experimental Results

Conclusions

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 5

Resource Model

Queuing System Generalized Processor Sharing (GPS) scheduler Request classes

Different arrival processes, service time distributions QoS Goal: Mean Response Time

GPS

Resource

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 6

Dynamic Resource Allocation

MONITOR

SystemMetrics

ResourceShares

APPLICATIONMODELS

ExpectedLoad

PREDICTORMeasuredUsage

ALLOCATOR

RsrcReqmts

RESOURCE

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 7

Dynamic Resource Allocation

ALLOCATOR

PREDICTOR

MONITOR

SystemMetrics

APPLICATIONMODELS

ExpectedLoad

MeasuredUsage

RESOURCE

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 8

Monitoring

Measure system and application metrics Queue lengths Request response times

Monitoring windows

AdaptationWindow

History

Time

MeasurementInterval

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 9

History AdaptationWindow

Prediction

Short-term prediction of workload characteristics Request arrival rate Average service time

Use history of measured system metrics

Mean

Last value

AR(1)

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 10

Prediction Accuracy

Prediction Error

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

20 40 60 80 100

AR(1)MeanLast value

No

rmal

ized

mea

n s

qu

are

erro

r

History Length (# measurement intervals)

Workload Prediction

200

400

600

800

1000

1200

0 20 40 60 80 100 120 140 160

ActualPredicted

No

. of

Req

ues

t A

rriv

als

TimeTime (min)

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 11

Dynamic Resource Allocation

PREDICTOR

MONITOR

APPLICATIONMODELS

ExpectedLoad

RsrcReqmts

ALLOCATOR

ResourceShares

RESOURCE

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 12

Measurement-based Model

Goal: Relate QoS metric to resource requirement

Idea: Model parameterized by online measurements Advantages:

Parameters do not need to be computed Allow adaptation to dynamic workload

Proposed: Transient Queuing System Description

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 13

Transient Queuing Model

Transient queuing behavior over adaptation window

Relation between mean response time T¯ and application share w

Little’s Law:

Relation is parameterized by the measured workload Arrival rate λ and mean service time s¯

])()0([)( tqtq

1

qCw

sT

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 14

Resource Allocation: Utility Model

Discontent function: Measures the QoS violations of an application

Constrained Optimization problem

u1

u2

Optimization

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 15

Constrained Optimization Formulation

Non-linear Optimization Problem:

Response Time

DiscontentDi

Goal

1i

iw

1min ii ww

i

iiw

TDMini

)( subject to

Solved using Lagrange multiplier method

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 16

Talk Outline

Motivation

System Model

Dynamic Allocation Techniques

Experimental Results

Conclusions

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 17

Experimental Setup

Simulation experiments

Soccer World Cup’98 Traces

Results based on a 24-hour portion of the trace 755,000 requests Mean req rate: 8.7 req/sec Mean req size: 8.47 KB

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 18

0

20

40

60

80

100

120

0 20 40 60 80 100 120 140 160

Res

ou

rce

Sh

are

(%)

Time (min)

Application 1

Application 2

Share AllocationWorkloads

0

20

40

60

80

100

120

0 20 40 60 80 100 120 140 160

Wo

rklo

ad

(%

sv

c r

ate

)

Time (min)

Application 1Application 1

Application 2

Total

Adaptation to Transient Overloads

Shares adapt to changing workload characteristics

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 19

Adaptation: System Discontent

0

5

10

15

20

25

0 20 40 60 80 100 120 140 160

Time (min)

Dis

cont

ent (

sec)

0

5

10

15

20

25

0 20 40 60 80 100 120 140 160

Time (min)

Dis

cont

ent (

sec)

GPS without reallocation GPS with reallocation

System Discontent is lowered substantially

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 20

Conclusions

Dynamic Resource Allocation needed for data centers

Measurement-based allocation: Monitoring and Prediction gather online state Use this state for application modeling and allocation

Future Work: Prediction policies Utility functions

http://lass.cs.umass.edu/projects/shop

UUNIVERSITYNIVERSITY OFOF M MASSACHUSETTS, ASSACHUSETTS, AAMHERST MHERST – – Department of Computer ScienceDepartment of Computer Science 21

Related Work

Prediction Statistical Prediction Models [Zhang00]

Application Models Queuing-Theoretic Models [Carlstrom02,Liu01] Control-Theoretic Models [Abdelzaher02,Lu01]

Data Centers MUSE [Chase01] COD [Moore02] Oceano [Appleby01]