Cost- and Energy-Aware Load Distribution Across Data Centers

Presented by Shameem Ahmed

Kien Le, Ricardo Bianchini, Margaret Martonosi, and Thu D. Nguyen (HotPower 2009)

Rutgers University and Princeton University

2

Motivations Large org has multiple Data Centers (DC)

Business distribution High availability Disaster tolerance Uniform access times to widely distributed client sites

Problems Consumes lots of energy Financial and environmental cost

How can we exploit the geographical distribution of DCs for optimizing energy consumption? 1. Different & variable electricity prices (hourly pricing)2. Exploit DCs in different time zones (peak/off-peak demand price)3. Exploit DCs located near sites that produce “green” electricity

2

Assumptions Multi-DC Internet services (e.g. Google, iTunes) DCs are behind a set of front-end devices Each service has single SLA (Service Level Agreement) with

customers SLA (L,P) = At least P% req must complete in <L time

Req can be served by 2 or 3 mirror DC Further replication increases state-consistency traffic

But no meaningful benefit in availability or performance

4

Is it True?

Is it True?

Contributions Framework for optimization-based request

distribution policy What % of client req should be directed to each DC Front-ends periodically solve optimization problem After % computation, front-ends abide by them until they are

recomputed

A greedy heuristic policy for comparison Same goal and constraints First exploits DC with best power efficiency Then exploits DC with cheapest electricity

5

Prior Research Energy management on a single data center A. Qureshi. HotNets 2008

Shut down entire data centers when electricity costs are relatively high

K. Le et al. Middleware 2007 Did not address energy issues, time zones, or heuristics

6

Request Distribution Policies

7

Principles and Guidelines Only minimizing energy cost is not enough

Must also guarantee high performance and availability

Respect these requirements by having the front-ends: Prevent DC overloads Monitor response time of DCs and adjust req distribution

accordingly

Each DC reconfigures itself by Leaving as many servers active as necessary + 20% slack for

unexpected load increase Other servers are turned off

8

“base” energy cost (servers are idle)

energy cost of processing the client req

Optimization Based Distribution (1/4) Problem Formulation

Policy EPrice: Leveraging time zones & variable electricity prices

Doesn’t distinguish DCs based on energy source

Doesn’t distinguish DCs based on energy source

Symbol Meaning

fi(t) % req to be forwarded to DC i

LT(t) Expected total # of req

Costi(t) Avg cost ($) of a req at DC i

BCosti(offeredi, t) Base energy cost ($) of DC i under offeredi load

LR(t) Expected Peak service rate (req/s)

offeredi LR(t) * fi(t)

LCi Load Capacity (req/s) of DC i

CDFi Expected % of req that complete within L time, given offeredi load

satisfied bemust SLA i.e. )(/)),()()((

1

0

PtLTofferedLCDFtLTtf

LCiofferedit

(t)ft

(t)fit

t

iii

t i

i

i

i

i

Optimization Based Distribution (2/4) Problem Formulation

Policy GreenDC: Leveraging DCs powered by green energy

10

energy cost of processing the client req “base” energy cost that is spent when active servers are idle

otheriwse ),(),( and )()(

GE consumpionenergy green if ),(),( and )()( i

tofferedbtofferedBCosttctCost

tofferedbtofferedBCosttctCost

iiiii

i

green

iii

green

i

i

i

Assumptions:1.DCs will increasingly be located near green energy source2.Green energy supply may not be enough to power DC entire period; Need backup (regular electricity)

Assumptions:1.DCs will increasingly be located near green energy source2.Green energy supply may not be enough to power DC entire period; Need backup (regular electricity)

Instantiating parameters Typical approach: front ends communicate & coordinate Proposed approach:

Each front end independently solves optimization problem LT(t), LR(t), and offeredi are defined for each front-end Load capacity (LC) of each DC is divided by # of front-ends CDFi instantiation

CDFi = Expected % of req that complete within L time Each Front end

Collects recent history of response time of DCi Maintains a table of <offered load, %> for each DC Similar table for BCosti: <offered load, base energy cost>

Symbol Meaning

fi(t) % req to be forwarded to DC I

Costi(t) Avg cost ($) of a req at DC I

BCosti(offeredi, t) Base energy cost ($) of DC i under offeredi load

LCi Load Capacity (req/s) of DC i

LT(t) Expected total # of req

LR(t) Expected Peak service rate (req/s)

offeredi LR(t) * fi(t)

CDFi Expected % of req that complete within L time, given offeredi load

Optimization Based Distribution (3/4)

11

Does this approach satisfy the constraints globally?Does this approach satisfy the constraints globally?

Solving Optimization Problem Electricity price prediction: Ameren Load intensity prediction: ARMA CDFi prediction: Current CDFi tables Can’t use LP solvers

Solving for entire day at once involves non-linear functions (e.g. BCosti, CDFi)

Use Simulated Annealing Divide the day into six 4-hour epochs

Solution recomputation (e.g. data center becomes unavailable)

Optimization Based Distribution (4/4)

12

Heuristic-Based Request Distribution (1/2) Cost-aware but simple For each epoch (1 hr), each front-end computes R = P x E

P = % of req must complete within L time (SLA) E = # of req front-end expects in next epoch (use ARMA) R = # of req that must complete within L time

Each front-end orders DCs that have CDFi(L, LCi)>= P

according to from lowest to highest ratio Remaining DCs are ordered by same ratio Concatenate two lists of DC to create final list (MainOrder)

13

),(

)(

ii

i

LCLCDF

tCost

Heuristic-Based Request Distribution (2/2) Request forward policy

Req are forwarded to first DC in MainOrder until its capacity is met

New req is forwarded to next DC on the list and so on After front-end has served R req within L time, it can

disregard MainOrder and start forwarding req to cheapest DC until capacity is met

What will happen if prediction is inaccurate? Adjusts R for next epoch

14

Optimization-based vs Heuristics-based

Characteristics Optimization (EPrice and GreenDC)

CA-heuristic

Accounting Period 1 day 1 day

Epoch length 4 hrs 1 hr

Load Predictions Per front-end for entire day Per front-end for next hour

Energy price predictions

Entire day Next hr

Recomputation decision

Epoch boundary Epoch boundary

Comm w/ DCs Yes Yes

15

Evaluation

16

Methodology (1/4) Implemented a simulator for large Internet service Simulate only a single front-end (East US) Front-end distributes req to 3 DC

17

Data Center Brown energy (cents/KWh)

Green Energy (cents/KWh)

Capacity (reqs/s)

DC1 (West US) 11.1 15.0 (solar) 125

DC2 (East US) 11.7 - 215

DC3 (Europe) 9.7 8.0 (wind) 125

Methodology (2/4) Request Trace

Day-long trace received by Ask.com

Trace doesn’t include response time To generate realistic DC response times:

Installed a simple service on 3 PlanetLab machines Req are made from a machine at Rutgers (front-end) Assumption: avg processing time of each req = 200 ms How to mimic effect of load intensity and network congestion

5% increase in response time for each 25% increase in load

18

ARMA

Methodology (3/4) Electricity Prices, Sources, and time zones

Three price scheme Constant rate, two rate (on/off peak), hourly prices

How to mimic different brown electricity prices for each DC? Shift default prices 3 hrs earlier or 6 hrs later

Assumptions Electricity price for Green DC is constant Green energy at each green site is enough to process 25% req

19

Ameren

Methodology (4/4) Other parameters

Assumptions A req consumes 60 J to process by 2 machines (including cooling,

conversion, and delivery overheads) SLA requires 90% of req to complete in 700 ms

Cost-unaware distribution policy Used for comparison basis Approach: similar to CA-heuristic

Orders DCs according to performance [CDFi(L, LCi)] from highest to lowest

Req are forwarded to first DC until its capacity is met New req are forwarded to next DC and so on

20

Effect of cost-awareness and pricing scheme (brown electricity) No cost for base energy

Result (1/4)

21

(1) Both cost-aware policies reduce costs even under constant pricing

(2) On/Off and Dynamic schemes reduce cost significantly

(3) EPrice always achieves lowest cost

EPrice: Optimization-based distribution (No green energy)

CA-Heuristic: Cost-Aware Heuristic (consider Costi/CDFi)

CU-Heuristic: Cost-Unaware Heuristic (consider CDFi)

EPrice: Optimization-based distribution (No green energy)

CA-Heuristic: Cost-Aware Heuristic (consider Costi/CDFi)

CU-Heuristic: Cost-Unaware Heuristic (consider CDFi)

Result (2/4)

22

Why does EPrice behave better than CA-Heuristic?

Data Center Brown energy (cents/KWh) Capacity (req/s)

DC1 (West US) 11.1 125

DC2 (East US) 11.7 215

DC3 (Europe) 9.7 125

Can Compensate DC3’s poor performance during future periods of low load.

Effect of green DC Considers only dynamic pricing Results are normalized against EPrice results w/ dynamic pricing No cost for base energy

Result (3/4)

23

Brown energy consumption is reduced by 35% by using green DC (3% cost increase)

Why do heuristic policies have higher cost

than GreenDC?

Result (4/4) Effect of Base Energy

Assumption: Server consumes power even when idle No DC consumes green energy

24

Base energy Cost savings

Conclusion Optimization framework for request distribution in multi-DC

To reduce energy consumption and cost To respect SLAs

Policies take advantage of time zones, variable electricity prices, and green energy

Propose a heuristic for achieving the same goal

Evaluation using a day-long trace from a commercial service

25

Discussions Only used 1 Front-end in experiment

More front-ends will satisfy global constraints?

How to ensure end-to-end QoS guarantee Can we combine SLA guarantee with QoS requirement provided by clients?

How to handle services with session state Soft state: Only lasts a user’s session with the service All req of a session must be sent to same DC

Can we apply the similar concept for multi-cloud structure? Optimize power Optimize monetary cost for online service provider

In multi-cloud computing, is it good to assume that data will be available in clouds beforehand? Pros and Cons

26