1

Power Management for Memory Systems

Ming ChenNov. 10th, 2009

ECE 692 Topic Presentation

2

Why Power Control for Main Memory?• Memory capacities have been increasing

significantly to accommodate CMPs.– CPU is no longer the only major power consumer.

• Memory is highly under-utilized.– Requirement on amount– Requirement on bandwidth

85.4%

92.2%

CPU & memory CPU : memory

1.21

0.69

3

Limiting the Power Consumption of Main Memory

Acknowledgments: The organization order and contents of some slides are based on Ricardo Bianchini’s slides.

Bruno Diniz, Dorgival Guedes, Wagner Meira Jr. Federa University of Minas Gerais, Brazil

Ricardo Bianchini Rutgers University, USA

4

Power Saving Vs. Power Control

• A problem with two sides– Trade-off between power and performance

• Power saving:– Guarantee performance first, then minimize

power.– Performance is primary.– Save energy bill.

• Power control: – Power capping: cooling, thermal, packaging, etc– Guarantee power budget first, then maximize

performance.– Power budget is primary.– Avoid system failure and thermal violations.

5

What is This Paper About?• Bunches of work done to save power.• The first paper I have read on power control for memory.• Propose 4 policies for Power Limiting (PL) in memory.

– Knapsack, LRU-Greedy, LRU-smooth, LRU-Ordered.• Combine Power Limiting with Energy Conserving (PL-EC).• Also provide performance guarantee. (PL-EC-Perf)

An interesting paper that combines the two sides of the power problem together.

6

Power Actuators

• RDRAM systems.– It is DRAM but not DDR SDRAM

• Each chip can be transitioned independently.• Different power states

7

Power Limiting

Memory Controller

chip1 chip3 chip4chip2

A SNS

8

Power Limiting

Memory Controller

chip1 chip3 chip4chip2

access

A SNS

9

Power Limiting

Memory Controller

chip1 chip3 chip4chip2

A SNS

10

Adjusting Power States

Memory Controller

chip1 chip3 chip4chip2

SNS N

11

Power Limiting

Memory Controller

chip1 chip3 chip4chip2

access

S SAN

Different approaches to adjust states.

12

Knapsack: Key Idea Multi-Choice Knapsack Problem (MCKP)

− Object: memory device.− Choices: multiple (power) states.− weight : the power consumption.− cost : the transition overhead to the active state.

Goal: Minimize the cost with the constraint of weight by putting each object in a state.

MCKP is NP-hard, which is solved off-line.

13

chip #

power stateKnapsack: An example

LRU queue is maintained for active devices. The LRU device is the victim. Switch the power states of the two chips.

14

LRU-Greedy• An LRU queue for all devices• If a device is to be accessed, move it to the tail and:

– Active? Go on …– Not active?

• Put the LRU to the shallowest state.• Still not? adjust the state for the next device.

15

LRU-Smooth• A LRU queue for all devices• If a device is to be accessed, move it to the tail and:

– Active? Go on …– Not active?

• Put the LRU to the next lower-power state.• Still not? adjust the state for the next device.

16

LRU-Ordered: Key Idea• An LRU queue for active devices• An ordered queue for devices in low-power mode

(shallowest first)• If a device is to be accessed:

– Active? Move it to the tail of LRU and Go on …– Not active?

• Move it from ordered queue to the tail of LRU queue.• Put the LRU to the top of the ordered queue.• Still not? adjust the state of the next device in the ordered

queue to the next lower-power state.

17

LRU-Ordered: An example

18

Energy Conservation (PL-EC)• If idle time in the current state > break-even time, then lower

power state.• Minimize delay*power2 by Knapsack for different # of

devices in the active state.• Whenever the # of active states is going to change:

– An active device is transitioned to the next low-power state when threshold expires.

– A low-power device is transitioned to the active state and it does not violate the budget

• The memory controller looks up the table and adjusts the states.

• If the activating device violates the budget, the basic scheme (PL) is used.

19

Performance Guarantee (PL-EC-Perf)

To what extent the energy to be saved? Basic strategy is from Xiaodong Li’s ASPLOS04 paper.

− 5M-cycle epoch− User-defined slowdown (3%) compared with PL− Compute slack at runtime.− If slack < 0, disable EC until the end of epoch

Disabling EC means reverting back to the corresponding PL policy.

20

Evaluation Methodology Single-core in-order CPU with integrated memory

controller Simics + memory subsystems OS and physical mapping of virtual pages are both

simulated. Memory system is driven by traces generated by Simics. Workloads: MediaBench, SPEC 2000, and client-server

applications. Memory size: 512 MB Performance is measured by the execution time of a trace

file.

21

Performance Vs. PL Polices

Knapsack and LRU-Ordered are best.

8 chips, 50% power budget.

22

Energy Vs. Policies

Compared with unrestricted execution. Unrestricted < budget bzip? (Critique)

8 chips, 50% power budget.

23

Performance Vs. Budget

8 chips under LRU-Ordered. Performance degradation is very small.

24

Energy Vs. Power Budget

Saving decreases as budget decreases. Uniform for all workload.

8 chips, 50% power budget.

25

Performance for PL-EC-Perf

8 chips, 25% power budget

LRU-Ordered 3% slowdown PD: an explicit

energy saving algorithm

PL/PL-EC-Perf almost works no worse than PD Exception is bzip2 ?

26

Energy Saving for PL/PL-EC-Perf

8 chips, 25% power budget

LRU-Ordered 3% slowdown PD: an explicit

energy saving algorithm

PL/PL-EC-Perf has more energy saving than PD. PD tends to send some chips to very deep states.

27

Conclusions Four power limiting policies are proposed.(PL) Performance degradation is surprisingly low. Limiting power + energy conserving (PL-EC) Limiting power + energy conserving + performance

guarantee (PL-EC-Perf) Limiting power consumption is as effective as doing

energy conservation explicitly.

28

A Performance-Conserving Approach for Reducing Peak Power Consumption in

Server Systems

Acknowledgments: The organization order and contents of some slides are based on Wes Felter’s slides.

Wes Felter, Karthick Rajamani, Tom Keller IBM Austin Research Lab

Cosmin Rusu University of Pittsburgh

29

Motivations

• System designers can no longer afford to accommodate peak power of all components (over-provisioning)

• System failures due to power overload and thermal violation.

• CPU is no long the only major power consumer.• CPU and the main memory share the same

power/cooling facility.

30

Anti-Correlation of Processor and Memory Power

• Processor and memory are not simultaneously highly utilized in workloads.

• Intuitively, the processor can’t keep itself and the memory busy0

10

20

30

40

50

60

0 10 20 30 40 50 60

CPU

Pow

er W

atts

Memory Power Watts

31

Unconstrained System Power

In theory, the system can use 83W.

32

What is This Paper About? Power shifting between processor and memory. The first paper that proposes the concept of

“power shifting”. Power estimation model based on the # of

activities. Propose 3 policies for power control in the

server level.− PLI, sliding window, and on-demand

33

Processor Power Model Power Vs. Dispatched

Instr./cycle 100K-cycle interval 28 applications Linear regression 9.3608 15.592y x

34

Memory Power Model

Power Vs. Bandwidth 100K-cycle interval 28 applications Linear regression 23.629 4.90y x

Pmem=#ranks*#devices*VDRAM*((Iactive- Iidle)*BW/ PeakBW+ Iidle)+ Pothers

35

Power Actuators

Power consumption correlates strongly with activity. Activity regulation techniques:

− Instruction decoding throttling− Clock throttling: effective duty cycles− DVFS

For processor:− Throttle at the instruction dispatch unit of the pipeline.

For memory:− Limit the total # of memory requests

36

Processor Core

SystemPower Controller

DispatchedInsns. Counter

Mem

oryM

emory

Fetch Throttling

Memory Controller

RequestCounter

RequestThrottling

Goes into powerdown mode when idle

Extensive clock gating

System Architecture

37

The # of activities is the same in the next interval. Estimate the power based on history. Allocate power based on estimates. Power allocation is enforced by thresholds of the # of

activities Activity-dependent power and standby power

Key Ideas

38

Power estimation− CPU power = C1*DPC0+C2− Memory power = M1*BW0+M2− Power to be allocated: Pdynamic = Pbudget – C2 – M2− DPC1 and BW1 for the next interval− Estimated active power: Pest = DPC0*C1+BW0*M1

Power allocation− DPC1= DPC0*Pdynamic/Pest

− BW1=BW0*Pdynamic/Pest

Threshold− Dth= DPC1*Period− Mth=BW1*Period

Proportional-Last-Interval Policy

39

Sliding window− Shorter interval is better for estimation accuracy.− Larger interval is better for reducing noise.− A larger window includes 20 intervals.

On demand− No violations, no throttling− Interval should be small enough.

Run-To-Exhaustion (RTE)− Power is monitored cycle-by-cycle.− Throttle when power is violated.− Impractical but provides a comparison.

Static: proportional to the peak power

Other Related Policies

40

Simulation Environment• Traces from hardware (SPEC) and Mambo

(e.g. JBB)• Integrated simulation environment

– Turandot+PowerTimer (Zhigang Hu et al., IBM TJ Watson)• Timing and power core model• 2GHz 970-like core w/ aggressive clock gating• 512KB L2 cache (power not simulated)

– + MEMSIM • Timing and power DRAM model• 4GB, 4 ranks of 128-bit 400MHz DDR (PC3200)

– Both simulators synchronized every cycle

41

Budget: 40 W Much better than static budgeting

PLI Vs. Static (1)

42

Budget: 50 W Average unconstrained power consumption < budget

PLI Vs. Static (2)

43

100K-cycle interval, 40W budget On-demand is generally the best.

− At the cost of at least one-interval budget violation

Policy Comparison

44

On-demand and the sliding window are generally the best. On-demand is even better than RTE for art.

− Not proactively throttle activities and at cost of short violations.

Normalized to RTE

45

Interval Size (PLI)

ammp: highly variable even at small interval and steady power.

Small interval has better fit for variations. Generally, highly application-dependent.

46

Critiques Paper 1

− Lack explanations for different workload (e.g. bzip).− Examples used for policies are not typical.− Do not explain why the performance is surprisingly slightly

degraded.

Paper 2− Open-loop estimation based scheme− Model verification− Power is not shifting but throttled− Very large performance degradation even budget is larger than

the average.

47

Comparison of the Two PapersLimiting Power Power Shifting

Target Peak power of DRAM system Peak power in the server level

Goal Peak power capping with energy conserving

Peak power capping

Methodology Knapsack optimization + heuristic

Open-loop estimation

Solutions A bunch of policies and comparison

A bunch of policies and comparison

Experiments Simulation Simulation

Power budget Larger than the average Larger than the average

48

Thank you !