Reduzierte thermische Widerstände: Schlüssel für eine … · Phase I (2012) SuperMUC at Leibniz Rechenzentrum Phase II (2015) 9288 IBM System x iDataPlex dx360 M4 – 43997256

© 2016 IBM Corporation Advanced Micro Integration [email protected]

Reduzierte thermische Widerstände:

Schlüssel für eine erfolgreiche Energiewende?

IBM Research – Zurich | Science & Technology

Patrick Ruch, Stephan Paredes, Brian Burg, and Bruno Michel

© 2016 IBM Corporation

IBM Research Zurich

Smart System Integration [email protected] 2

Why are Breakthrough Innovations Needed?

• Past decades: Get access to low cost energy

– Get access to fission energy

– Get access to chemically stored solar energy (coal, oil, gas)

– Get access to low cost, low efficiency solar energy

– No focus on exergy and efficiency Exergy waste >75%

• Present: “Energy turnaround”

– Reason risk for human survival (nuclear or climate catastrophes)

– Major strategy change required from getting access to energy

to getting access to exergy and to improve efficiency

– Energy sector is mature and there is no breakthrough innovation possible ??

• Future decades: Use energy efficient and minimize waste

– Focus on efficiency allows thermal wastes to be reduced 3x to < 25%

– Efficiency: IT to reduce our energy consumption by better organization and analysis

– Efficient solar technologies with highest efficiency and minimal radiative forcing

– Efficient thermal transformers reduce exergy waste from 70% to <20%

– Better thermally mediated energy conversion processes

• Key breakthrough innovation

– Disruption when several rapid evolving elements meet and lead to a new convergence


IBM Research Zurich


Why Innovation from Outside of Energy Industry?

• Huge recent focus on cooling in computers

– Huge power densities AND exceptionally low thermal budget

– Exceptionally fast development 1000x overall demand increase in 10 years

– 30 x due to efficiency and 30x higher energy demand +40% per year!

– ICT industry has 3% ww carbon footprint larger than airline industry!

• Exceptional power density in computers (areal and volumetric)

– Increase from 1 to 100 W/cm2 = 0.01 to 1 MW/m2 at a gradient of <10ºC in 20 years

– Volumetric power density in computer is moderate ~0.01 W/cm3 (0.01 MW/m3)

– R&D goal: Dense computing massively improves efficiency (5000x)

– But requires 100-1000 W/cm3 system level power density

• Huge R&D investment to improve areal and volumetric power density

– Cooling technology to handle 1000 W/cm2 or 10 MW/m2 (at a gradient of 50ºC)

– Interlayer cooling at 1 kW/cm3 = 1GW/m3

– >10x better thermal performance than any other energy conversion technology

– Much larger power density than typical power conversion

– 50 kW piston engine (1l) = 50 W/cm3 for working space and ~1W/cm3 for system

– Reactor or furnace 300 MW in 300m3 = 1 MW/m3 or 1 W/cm3 <<1 W/cm3 for system

• IT Industry – Energy Industry Convergence

– Convergence drives disruptions in both industries


IBM Research Zurich


How are Thermal Resistances Massively Reduced?

• Transition from Air cooling to liquid cooling in computers

– Water with 4000 times better heat capacity and 30 time better thermal conductivity

• Microchannels

– High aspect ratio massive surface enlargement

– Disadvantage laminar flow and large pumping power

• Concept of branched hierarchical transport

– Our blood circulation system reaches best mass transport with minimal pumping power

– Optimal branching factor

• Ultra-short microchannels

– Prevent established laminar flow

• Radical miniaturization and use of silicon (dioxide) as structural material

– Good thermal conduction

– No thermal interfaces needed for heat flow

– Radical miniaturization

• Better interfaces

– Filling materials/gaps with percolation

– Improved overall thermal conduction due to necking


IBM Research Zurich


Entry into Energy Research: High-performance Liquid Cooling Datacenter-level power consumption

cooling infrastructure

APC Whitepaper #154

Rev. 2 (2010) Water-cooled IBM blade servers (2010)

HS22

QS22

IBM BladeCenter QS22/HS22 Cluster @ ETH Zurich

Wate

r-coolin

g m

odule

Connected to ETH hot water grid

Water- vs. air-cooling

Air-cooling Water-cooling

Thermal

conductivity

[W/(m∙K)]

Volumetric

heat capacity

[Wh/(L∙K)]

Air 0.02 0.0003

Water 0.6 1

qRT th

thermal gradient thermal resistance

heat flux

Rth = 1 Kcm2/W

T = 50-100 K

For 50-100 W/cm2:

Rth = 0.1 Kcm2/W

T = 5-10 K

First hot-water cooled supercomputer

with energy recovery

•80% energy recovery @ 60°C

•Direct feed to space heating grid

•Energy consumption -40%

•CO2 footprint -85%


IBM Research Zurich

Smart System Integration [email protected] IBM Zurich

Research

Laboratory |

7-Nov-16

© IBM

Research

2009

6

Manifold Micro-Channel Heat Sink

W. Escher, T. Brunschwiler, B. Michel and D. Poulikakos, “Experimental Investigation of an Ultra-thin Manifold

Micro-channel Heat Sink for Liquid-Cooled Chips”, ASME J. of Heat Transfer, 132, 081402-10 (2010).

W. Escher, B. Michel and D.

Poulikakos, “A novel high

performance, ultra thin heat

sink for electronics”, Intl. J.

Heat and Fluid Flow 31,

586-598 (2010).


IBM Research Zurich


Datacenter: Cooling Infrastructure Chillers

(Refrigeration)

Evaporative Tower Fans

Condenser

Chilled

Water

CRAC/

CRAH

s

Racks & Fans

Electrical Power


IBM Research Zurich

Smart System Integration [email protected]

Hot-Water-Cooled Zero Emission Data-

centers „Aquasar“

CMOS 80ºC

Water In 60ºC

Micro-channel

liquid coolers Heat exchanger

Direct „Waste“-Heat usage

e.g. heating

Biological inspired: Vascular systems optimized for low

pressure transport

Water Out 65ºC

Water Pump

8


IBM Research Zurich


Zero-Emission Data Centers

• High-performance chip-level cooling

improves energy efficiency AND

reduces carbon emission: – Cool chip with DT = 20ºC instead of 75ºC

– Save chiller energy: Cool with T > 60ºC hot water

– Re-use: Heat 700 homes with 10 MW datacenter

• Need carbon footprint reduction – EU, IPCC, Stern report targets

– Chillers use ~50% of datacenter energy

– Space heating ~30% of carbon footprint

• Zero-emission concept

valuable in all climates – Cold and moderate climates:

energy savings and energy re-use

– Hot climates: Free cooling, desalination

• Europe: 5000 district heating systems – Distribute 6% of total thermal demand

– Thermal energy from datacenters absorbed IBM Research – Zurich, Advanced Micro Integration [email protected] 9


IBM Research Zurich


Power Dissipation

up to 3.6 MW

HPLinpack Performance

2.9 PFLOPS

IDPX DWC dx360 M4

9288

Power Dissipation

up to 1.3 MW

HPLinpack Performance

X.X PFLOPS

NXS DWC nx360 M5

3096

Power Usage Effectiveness

PUE 1.1

World’s

Most Powerful &

Energy Efficient x86 Supercomputers

Phase I (2012)

SuperMUC at Leibniz Rechenzentrum

Phase II (2015)

9288 IBM System x iDataPlex dx360 M4

– 43997256 Components

– 8.08 m2 CMOS 4.22x1013 transistors

– 74304 4 GB DIMMs

– 11868 IB Fibre Cables 192640 m

– 34153 m Copper tubes, 7.9 m3 Water

– 18978 Quick Connects

Mass 194100 kg

IBM System x / Lenovo NeXtScale

DWC nx360 M5

• 90% heat flux to warm water

• 40% less energy used than air-cooled


IBM Research Zurich


Leibniz-Rechenzentrum (2015) Largest installation utilizing

datacenter waste heat

for adsorption cooling

APC Whitepaper #154 Rev. 2 (2010)

Warm-water cooling +

adsorption heat pumps:

-50% power consumption

for non-IT components

Value proposition without

reliance on external demand

11

Energy Efficiency in Datacenters


IBM Research Zurich


• Water-cooling used to be a necessity

• Today, water-cooling supports energy efficiency

and computing performance

• Enable ultra-dense systems: Computing as a service

• Ecosystem around hot water cooling and heat re-use

Water cooling

system

Energy consumption

reduced by

up to 40%

Direct reuse of waste heat

cuts CO2 emissions by

up to 85%

Microservers

for cloud

Computing

Water-Cooled Computing Roadmap


IBM Research Zurich


Scaling to 1 PFlops in 10 Liters

System with

1 PFlops in

10 liters

P. Ruch, T. Brunschwiler, W. Escher, S. Paredes,

and B. Michel, “Towards 5 dimensional scaling:

How density improves efficiency in future

computers”, IBM J. Res. Develop. 55 (5,

Centennial Issue), 15:1-15:13 (2011).

• Efficiency comparison – 1PFlops system currently consumes ~10MW

– 0.1 PF ultra-dense system consumes 20 W

• Ultra-dense Bionic System – Stack ~10 layers of memory on logic

– Stack several memory-logic stacks to stack of stacks

– Combine several blocks of stacks to MCM (MBM)

– Combine MCMs to high density 3D system

• Key enabling technologies – Interlayer cooling

– Electrochemical chip

power supply

• Impact – 5’000x smaller power

– 50’000’000x denser

– Scalability to zetascale With cooling


IBM Research Zurich


Experimental Smarter Energy Research Agenda

High performance

microchannel coolers

Growth in Cloud

and Big Data

Heat exchanger

Pump

Underfloor

heating

Research background

Economic value of heat reduces datacenter total

cost of ownership by 50-70% lower energy cost

60°C

65°C

>700 W/cm2

leverage in

Critical energy & environmental issues

Sustainable generation

Electricity and heat

Fresh water scarcity

Renewable heating and cooling

Zero-emission

datacenter

High-concentration

PV/thermal

Adsorption

heat pump

Membrane distillation

desalination

Electrochemical redox

energy conversion


IBM Research Zurich


High Concentration PV/thermal (HCPVT) Multigeneration

•Solar provisioning of electricity and heat

today typically employs two separate

power stations → doubled cost

•Current solar systems capture <35% of

incoming solar irradiation → >65% waste

HCPVT system captures >80%

of solar energy content with

one installation


IBM Research Zurich


Better Competitiveness and Efficiency by Integration

• Matching provider and consumer of heat

• Making use of optimized interfaces

• Best Solar to Electricity, Water, Cooling

Efficiency

Low grade heat thermal process


IBM Research Zurich

Smart System Integration [email protected] 17 17

Advantages of HCPVT System

• Higher concentration factor

– Air cooling looses 50ºC in package and wastes heat

– 10x reduced package thermal resistance allows

– 90ºC fluid removes heat from 100ºC cells for >2000 suns

• Higher electrical efficiency

– PV, CPV, and CSP have overall system yield <30%

– 350ºC solar thermal systems dissipate 75% waste heat

– High concentration allows triple junction PV with >30% yield

• Desalinated water, heating, and cooling from waste heat

– HCPVT makes heat re-usable without

degrading PV efficiency

– System output 80%

– Electrical power, desalinated water and

cooling at lower cost

• Superior HCPVT system performance

– >50% heat available on top of

the electrical yield

NASA SSE Release 6.0 Data set 22-year Monthly &

Annual Average (1983 - 2005). Map by DRL (2008)

Heating / Hot water

4.4 kWh/(m2 day)

Yield 50%

Electrical Power

2.6 kWh/(m2 day)

Yield 30%

Desalination

>50 L/(m2 day)

GOR 7-8

Cooling

2.8 kWh/(m2 day)

COP 0.6

OR

OR

AND

HCPVT

Irradiance 850 W/m2 or

8.8 kWh/(m2 day)


IBM Research Zurich


HCPVT Technology Advantage

PV chip with inter-

mediate substrate

PV chip direct attach

Multi Cell Module

(MCM)

PV chip direct attach

Single Cell Module

(SCM)

TIM 2

TIM 1

PV Cell

heat sink Manifold

TIM 1

PV Cell

1st level

manifold Si cooler chip

assembly

Large Multi Cell Receiver (MCR)

1500 suns

12 kWel / 25 kWth prototype units in 2015

Receiver packaging and thermal management

10x reduced thermal resistance with

respect to air-cooling

TIM: Thermal Interface Material

• 40m2, 12kW electrical and 25 kW thermal

• Cooling water 90°C

• 30% electrical, 50% thermal, and 80% total

• LCOE: <0.1$/kWh for sunny locations

• 10x lower cost than steel/glass

technologies

• Reduce the number of expensive multi-

junction solar cells

• Combination with adsorption cooling

and desalination


IBM Research Zurich


Are Active or Passive Solar

Technologies better Counter-

measures to Global Warming?

• White roofs reduce urban temperatures

for cities in the Sunbelt while solar panels cause a heat island effect

An inconvenient truth for the cost focused solar industry

Need refocus on efficiency; avoid building integration

?


IBM Research Zurich


Large Energy Demand in Sunny Regions

• Tripled global cooling demand (2010 -2040)

predicted by International Energy Agency (IEA)

• Driven by growth markets in hot regions

• Vapor-compression cooling strains power grid

• Solution: Solar thermal cooling with afternoon

peak output when demand is highest

• HCPVT with heat driven sorption chillers

provides cooling without loss of electrical output

• Smallest solar fields – maximized yield

• No water consumption: Dry cooling, no cleaning

Off Peak Base Load Off Peak

Peak with

compr. cooling

Peak with

sorption cooling


IBM Research Zurich


Low temperature

Key performance metrics

• Specific cooling power (SCP) [W/kg] Cooling power per unit adsorber mass

• Coefficient of performance (COP) [-] Cooling power per unit thermal driving power

→ Adsorbent materials

→ Heat and mass transfer

→ Heat exchanger and system design

High temperature Medium temperature

Adsorption Compression

Water-based Refrigerants

COPel >10 COPel <5

1500 $/kW 500 $/kW

30 W/kg 100 W/kg

10 kW/m3 60 kW/m3

Today’s technology

Adsorption Cooling


IBM Research Zurich


Transport dynamics Adsorbents

Refrigerants

Silica gel Zeolite

Activated

carbon

Alumino-

phosphates

MOFs Salt

composites

Sources: Henninger et al., JACS 131 (2009) 2776; Critoph & Zhong, Proc. IMecE 219 (2005) 285; Freni et al., Appl. Therm. Eng. 27 (2007) 2200; Aristo

Name Formula Boiling point

[°C]

Latent heat

[kJ/kg]

Latent heat density

[kJ/m3]

Water H2O 100 2 258 2 163

Methanol CH3OH 65 508 872

Ammonia NH3 -34 1 368 932

Sulphur dioxide SO2 -10 605 534

Adsorption Cooling


IBM Research Zurich


Characterization of Adsorption Dynamics

scale bar: 1 cm

Cooling power measurement Experimental setup


IBM Research Zurich


Physical modeling of

heat & mass

transport with

sorption phenomena

Location-specific modeling of solar thermal and

waste heat driven cooling systems 1 2

Synthesis of adsorbents with tailored

application-specific properties 3

Characterization via vapor sorption

isotherms and adsorbate diffusion

coefficient measurements 4

Adsorbent assembly for optimized

heat & mass transport rates 5

In situ transient thermography and

cooling power characterization 6

Test facility for 1 kW heat

exchanger

characterization 7

0.5 mm 100 µm

0

10

20

30

40

50

60

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

Relative pressure

Ma

ss u

pta

ke

[%

] M

ass u

pta

ke [%

]

Relative pressure

Domestic hot water

150°C | 60°C | 12°C

Space heating

80°C | 40°C | 12°C

Air-conditioning

60°C | 30°C | 18°C

Summary of Expertise


IBM Research Zurich


Thermally Driven Heat Pumps in Switzerland: National Project

Scenario assessment

Impact evaluation

Technology development

Sub-project 4 System & interface

Sub-project 5 Sustainability analysis

Sub-project 1 Tailored materials

Sub-project 4 System & interface

Sub-project 5 Sustainability analysis

Sub-project 2 Advanced adsorbers

Sub-project 3 Compact heat pump

THRIVE Project leader

NRP 70

http://www.psi.ch/

http://www.psi.ch/


IBM Research Zurich


Why now?

• Why nobody noticed this fact so far

– No overlap between the two disciplines

– Insufficient focus on efficiency

• 1) Because of past focus on energy and not exergy/efficiency

– Waste heat was not an issue BUT need to turn your enemy into your friend

– Paradigm change in computer industry from performance focus

– (squeeze as much as you can out of a silicon processor chip) to

– Efficiency focus (squeeze as much as you can out of a Joule)

• 2) Because of our focus on electrical energy

– Initial energy transport was done by heat and then by electricity due to introduction of

electrical transformer

– Heat requires understanding of the concept of exergy: Need thermal transformer

• 3) Because of lack of technology and effort

– Focus on electrically driven heat pumps

• Energy turnaround triggers a lot of research

– NRP 70 and 71 programs in Switzerland

– THRIVE project

– Reduced convective and conductive thermal resistance everywhere


IBM Research Zurich


Thermal Resistance: Importance beyond Heating/Cooling

• Major reason for limited efficiency

– Losses due to convective and conductive thermal resistances

• Convective thermal resistance

– Microchannel (disadvantage laminar flow)

– Hierarchical branched transport network (developing flows)

• Conductive thermal resistance

– Better materials copper

– Shorter path lengths

– Better interfaces

• Most energy conversion processes use thermal path or cause waste heat

– Heat driven heatpump: less than 30% exergetic efficiency

i.e. COP of 0.6 for Tdrive 90ºC, Tamb 30ºC and Tcool 10ºC 20% exergetic efficiency

– Steam engines 42% = 58% losses (mainly in boiler and due to Rankine Cycle)

– CHP 65% = 35% losses

– Solar thermal: <15%

– Photovoltaic: < 15%

– Combined CPV and thermal 80% total and >50% exergetic efficiency


IBM Research Zurich


Summary

• Reuse ICT heat for heating and cooling Zero-emission datacenter

– Clear technology lead >10x lower thermal convective resistance than in energy industry

• Convert heat from Solar HCPVT and datacenters into cooling

– Needs high efficiency HCPVT receiver and district cooling

– Heat driven heat pump 10x better due to lower conductive resistance and better isotherm

– Heat pump with high exergetic efficiency Thermal Transformer

• Move solar industry away from the wrong track

– Low efficiency solar contributes to global warming by radiative forcing

– Solar in sunbelt: No building integration and high efficiency needed

• Microchannel flow boiling to be used in low grade heat steam engines

– Rankine cycle with limited efficiency Should use near isothermal expander

• Common part: Massively reduced convective and conductive heat transfer


IBM Research Zurich


Summary: Solving the Energy–Water–Cooling Challenge

• Solutions for Fresh water solar dilemma and large energy and cooling demand

• Selection of emerging technologies with a huge potential

– Emerging to become mature as part of a project roadmap

– Offer higher overall efficiency and zero carbon footprint during operation

– Best overall sustainability profile

• Each technology alone able to displace the current market leader

• Energy HCPVT provides best LCOE values

• Water Heat driven membrane distillation desalination

• Cooling Efficient heat driven adsorption cooler

• Best Solution: Amplify overall efficiency by integration

Making use of optimized interfaces

• Why thermal processes? Efficiency and low cost storage

• IBM experience in heat reuse, higher efficiency, and power density

• Initial prototype solutions available for testing in 2015

• Breakthrough solutions for sustainable solar energy, cooling, and water

• Project roadmap for 2016 and 2017 for complete low cost, high performance

solutions


IBM Research Zurich


Heat Transformation Through Solid Sorption


IBM Research Zurich


Green Datacenter Market Drivers and Trends

• Increased green consciousness, rising cost of power

• IT demand outpaces technology improvements – Server energy use doubled 2003-2008; temporary slowdown due

to economic crisis; resumed growth is not sustainable

– Koomey Study: Server use 1.2% of U.S. energy

• ICT industries consume 2% world wide energy – Carbon dioxide emission like global aviation

Real Actions Needed

Brouillard, APC, 2006

Future datacenters dominated by energy cost;

half energy spent on cooling Source IDC, 2009

IBM Research – Zurich, Advanced Micro Integration [email protected] 31


IBM Research Zurich

Smart System Integration [email protected] IBM Zurich

Research

Laboratory |

7-Nov-16

© IBM

Research

2008

32

Chip Scale Liquid Cooling

A

A’

A-A’

Arrayed jets, distributed return

SEM cross-section of two-level

jet plate with diameter of 35µm

Biological vascular systems are optimized for

the mass transport at low pressure

Direct Liquid Jet-Impingement Cooling with Micron-Sized

Nozzle Array and Distributed Return Architecture, T.

Brunschwiler et al., Proceedings ITHERM (2006)

Cooling of up to 350 W/cm2


IBM Research Zurich


Adsorption Cooling Energy Ecosystem

Renewables

Solar thermal

Driving heat

Heat rejection

CPV/T

Geothermal

Combined heat & power

Gas turbines Micro-CHP

Cooling

Waste heat

Industry

• Residential

• Commercial

• Industrial

• District cooling


IBM Research Zurich


Low / Medium Grade Heat Driven Processes

• Desalination needed for increasing demand and diminishing supply of water

– Multistage Flash and Reverse Osmosis overstresses electrical supply / grids

– Temperatures limited due to scale formation – fits with CPVT temperatures

• Multi Effect Membrane Distillation (MEMD)

– Uses micro-porous, hydrophobic membrane, low grade heat and plastic vessels

– Combines advantages of RO with water purity from distillation processes

MEMD and CPVT with best output of electricity and desalinated water

– Water needed during hot and dry season with lots of sun

• Compression cooling for air conditioning overstresses electrical grids

– Multi-generation supporting space heating and cooling via an adsorption chiller

– IEA predicts tripling of cooling demand from 2010 to 2040

Adsorption and CPVT provide cooling on top of electrical power

– Heat is available at low cost in CPVT system with peak power around noon

– Cooling needed during hot season with lot of sun

Solar heat driven cooling and

desalination solve energy and water problem

Documents

Reduzierte thermische Widerstände: Schlüssel für eine … · Phase I (2012) SuperMUC at Leibniz Rechenzentrum Phase II (2015) 9288 IBM System x iDataPlex dx360 M4 – 43997256