Energy- and Thermal-Aware Design of Many-core Heterogeneous Datacenters

David Atienza, Luca Benini, Edouard Bugnion, Babak Falsafi, Marcel Ledergerber, Fabrice Roudet, Patrick Segu, Lothar Thiele and John Thome

2016 Nano-Tera.ch Annual Meeting

Datacenter Energy Trends Not Sustainable

2

Bill

ion

Kilo

wa

tt h

ou

r/ye

ar

2001 2005 2009 2013 2017 0

40

80

120

160

200

240

280Datacenter Electricity

Demands In the US [Energy Star]

A Modern Datacenter (DC)

Density per rack: 8kW-15kW

[IDC]: Mega DCs is

70% of all DCs in 2018

Change energy increase trend, how to “flatten” it?

Modern datacenters (DC) increase power demands 20 MW

Power is beginning to clearly dominate costs in datacenter management

In Switzerland, 3-4% of all electricity, growing at >20%

Swiss industry is heavily based on services and requires significant IT support

Approach: System-Level Energy-Efficient Integration

3

Multi-scale system engineering to cover complete datacenter re-design Balanced industry-academia participation: 3 large Swiss-based industrial partners

Evaluation of proposed approaches with experimental prototypes

ESL-EPFL

IIS-ETHZ

DCSL-EPFL

PARSA-EPFL

TIK-ETHZ

LTCM-EPFL

WP1: Ultra-Adaptive Server Technologies

4

Analytical models for 1st-ever 4-core in 28nm RVT FDSOI [Solid-State Elect.‘16]

Physically validated energy, performance and temp. confirm energy-scalability of the approach

Evaluated advanced power management to compensate process and temperature variation

28nm FD-SOI

Very effective compensation of leakage and dynamic

power in complete temperature range by VBB or VDD

WP2: Energy-Scalable Server Architectures

Adaptive multi-core design with FDSOI [DATE’16]:

Efficient operation between 200 MHz and 2.5 GHz

Fast change of operating V/F point

Services response time not requiring peak processing

Scale-out loads can operate at 200-500 MHz

VM loads can operate at less than 1 GHz

5

Near-threshold (FDSOI) enables

scalable many-core servers

log(mW)

1GOPS/mW

40nm

log(GOPS)

28nm FD-SOI

Goal: reduce «bending up» of Server’s energy curve

28nm

QoS Limit

WP3: Energy-Free Cooling

6

Built low-height thermosyphon prototype 15-cm prototype proves rack compatibility

Effective use of gravity to remove heat

Mini-thermosyphon demonstrator built [ITHERM’16]:

Validated approach for a 2U server blade,

design done for HP blade (donated by Yahoo)

WP4: Integrated Datacenter Control

Multi-objective controller for geo-distributed data centers Global phase: Virtual Machines (VMs) clustering

Local phase: VMs allocated using CPU-load and data correlation

Green controller: on-line management of energy sources

Significant improvements in cost, energy and perf. Up to 15% in energy and 35% in costs ($$$)

Global phase

(VMs clustering)

Local phase

(VMs

allocation)

Green

controller

DC 1

Local phase

(VMs

allocation)

Green

controller

DC N

Scalable approach tested with more than

12’000 servers distributed in 3 data centers

7

Server and Rack Management Prototype Built with Eaton

Installation at Eaton facilities in Le Lieu Real-time monitoring and management

Reduce peak temp. on multi-rack system

Developed architecture compatible with Eaton’s servers and racks products Evaluations with larger IT setups on the way

8

Remote access possible for testing of

new YINS management strategies

Conclusion

Successful development of prototypes to validate YINS’ research ideas Validated use of 28nm FDSOI tech. to build energy-scalable server architectures

Validated low-height thermosyphon passive cooling system

Developed multi-objective controller for virtualized geo-distributed datacenters

Future work: continue research on inter-WP cross-layer optimizations WP1: adaptive server technology with integrated power and cooling delivery

WP2: optimize memory hierarchy (DRAM, caches, etc) and OS customization

WP3: develop micro-thermosyphon demonstrator in HP blade

WP4: expand experimental validation of multi-rack and datacenter power management

9