GREEN COMPUTING

CS 595

LECTURE 12

4/1/2015

Number one concern of large scale data centers?

• Efficiencyaffects:

accessibilityreliabilitysustainabilitycosts

Can we affect datacenter efficiency with software?- yes!

OUTLINE• Power consumption in large datacenters

• Alternative energy sources

• Green hardware design best practices

• Server level hardware

• DC/DC, AC/DC conversions

• Rack power distribution

• UPS

• Cooling

• Building power distribution

• Green software design best practices

• Resource virtualization

• Storage

• Computational

POWER CONSUMPTION IN LARGE DATACENTERS

• United States: 91 Billion kWh (2013)

• Datacenter power usage effectiveness (PUE):

• ESIS – Consumption for supporting infrastructure

• EITS – Consumption for IT power substations

• ETX – Building Transformer losses

• EHV – High voltage cable losses

• ELV – Low voltage cable losses

• EF – Consumption from fuel (natural gas, oil)

• ECRAC – CRAC consumption

• EUPS – Loss at UPS

• Enet1 – Network room energy from type 1 unit substitution

• PUE?

• For every (PUE)Watt consumed at the datacenter, 1 Watt is delivered to the IT load

PUE =

•2014 Global Datacenter average PUE: 1.7

• Amazon: 1.45

• Microsoft: 1.13

• Google: 1.1

• Yahoo: 1.07

• Facebook: 1.07

POWER CONSUMPTION IN LARGE DATACENTERS

• Datacenter energy breakdown:

• Computing: 52%

• CPU – 15%

• Server Power supply – 14%

• Other Server Components – 15%

• Storage – 4%

• Networking – 4%

• Support Systems: 48%

• Cooling – 38%

• UPS – 5%

• Building Power Distribution/Transformer – 3%

• PDU – 1%

• Lighting – 1%

POWER CONSUMPTION IN LARGE DATACENTERS

• Average and maximum power densities:

• 10 years ago:

• 100W – 175W per sq ft.

• Modern data center:

• Average: 225W

• Maximum: 400W

• Previous slide: 5,000 sq ft. datacenter

• 2mW of energy to dissipate!

• Is this a large datacenter?

• 2011: Clemson University completes its new datacenter:

• 50,000sq ft.

• 25,000sq ft. -office space

• 25,000sq ft. - rack space

• 4.5mW at full capacity (current)

ALTERNATIVE ENERGY SOURCESSOLAR

• Solar Constant: amount of incoming solar electromagnetic radiation

• ~1.36 kW/m2 (maximum)

• Average solar panel efficiency: 14%

• 19w/ft2 (best condition)

• Closer to 10w/ft2

• 5,000 sq ft. datacenter:

• 2mW 200,000 sq ft of solar panels! (4.59 acres) lol?

• Semprius (solar startup): 33.9%

• 1.89 acres

• Energy storage?

• Datacenter placement?

• Cloudy?

ALTERNATIVE ENERGY SOURCESWIND

• Advantages:

• Generate 250W – 1.8MW per turbine

• Green, renewable energy

• Disadvantages:

• Unpredictable

• Noise

• Aesthetics

ALTERNATIVE ENERGY SOURCESGEOTHERMAL

• Cooling, Cooling, Cooling!

• Approach 1:

• Climate cooling

• Strategic placement of datacenter that allows outside air temperature to help regulate inside temperature.

• Approach 2:

• Ground cooling

• Average temperature at 20ft below earth’s surface: 54F

• Datacenters can use closed loop venting ducts underground to dissipate the heat produced by equipment.

GREEN HARDWARE DESIGN BEST PRACTICESSERVER LEVEL HARDWARE - CPU

• CPUs with lower TDP:

• Intel Ivy Bridge Xeon:

• 22nm manufacturing process

• E3-1280 – 4 core w/HT (8 threads)

• Max TDP: 69W

• E5-2600 V2 – 10 core w/HT (20 threads)

• Max TDP: 130

• AMD Piledriver Opteron:

• 32nm manufacturing process

• 6366 HE – 16 core

• Max TDP: 85W

• 6386 SE – 16 core

• Max TDP: 140W

• Goal: Increase Flops/Watt

GREEN HARDWARE DESIGN BEST PRACTICESSERVER LEVEL HARDWARE - CPU

• Performance per watt: rate of computation that can be delivered by a CPU for every Watt of power consumed.

• www.green500.org:

• #1: 5.27 GFlops/W

• #1 - #10 average: 3.91 GFlops/W

• #500: 22.11 MFlops/W• Is this green?

• Google Nexus 6: ~100 MFlops/W• CPU + GPU + Android System

GREEN HARDWARE DESIGN BEST PRACTICESSERVER LEVEL HARDWARE - STORAGE

• HDD vs SSD:

• WD Caviar Blue 500 GB SATA 6GB/s

• Read/Write: 6.8W

• Idle: 6.1W

• Read/Write performance: 130MB/s

• Seagate Cheetah 15k 146GB SAS

• Read/Write: 17.2W

• Idle: 14.4W

• Read/Write performance: 186MB/s

• OCZ Vertex 3 240GB SATA 6GB/s

• Read/Write: 1.25W

• Idle: 0.53W

• Read/Write performance: >500MB/s

• Cost per GB?

• HDD: ~$0.05/GB

• SSD: ~$0.47/GB

GREEN HARDWARE DESIGN BEST PRACTICESSERVER LEVEL HARDWARE - RAM

• DDR2 vs DDR3

• DDR2 – 1.8v

• 1066 MT/s

• 80nm manufacturing process

• DDR3 – 1.25v – 1.65v

• 2133 MT/s

• 20nm (Samsung, 2012)

• DDR4 (September 2012):

• 1v – 1.2v

• Up to 4266 MT/s (megatransfers/sec)

• 20nm current manufacturing process

GREEN HARDWARE DESIGN BEST PRACTICESSERVER LEVEL HARDWARE - MOTHERBOARD

• 85% of power consumed by motherboard:

• Voltage Regulators

• Converts voltage provided by power supply into appropriate voltages needed by the CPU, RAM, Chipsets, etc.

• Also used to “clean up” DC voltage from the power supply.

• Linear voltage regulator:

• Solid state semiconductor to regulate voltages

• ~60% efficient

• Switching voltage regulator:

• Circuit that transitions between full regulated out, low-current output, and off

• Generates less heat ~95% efficient

• Use motherboard with switching voltage regulators for higher efficiency.

GREEN HARDWARE DESIGN BEST PRACTICESSERVER LEVEL HARDWARE - NETWORK

• Use of power efficient networking components:

• Switches

• Routers

• NICs

• Devices can be scaled depending on demand

• Device low power mode

• Turn off ports

• Goal:

• Increase Bandwidth/Watt in communication components

GREEN HARDWARE DESIGN BEST PRACTICESAC/DC, DC/DC CONVERSIONS

• AC/DC conversions:

• Translates non-polarized alternating current (building power feed) to polarized direct current (used by IT components)

• Not 100% efficient

• Heat generated equates to loss

• Also must be dissipated requiring more power consumption!

• AC/DC conversion locations:

• Building feed to UPS

• Server Power supply

• EPA Energy Star rating – 80% efficiency across all rated power output.

• Switching voltage regulator (previous slides)

GREEN HARDWARE DESIGN BEST PRACTICESRACK POWER DISTRIBUTION

• Problem with current rack deployment?

• Rack mount servers connected to UPS

• Power transforms at UPS and again at the server

• Solution (Google):

• Remove unneeded AC/DC, DC/AC transformations

• AC from building line feed connects to UPS in the rack

• UPS performs AC/DC transformation

• Servers connect directly to UPS’s DC output

GREEN HARDWARE DESIGN BEST PRACTICESUPS

• Uninterruptible Power Supply:

• Stores power to ensure IT component operations in the case of short power outages

• Also used to “clean” electricity from outside source

• Efficiency: 92% - 95%

• Arranged in a 2N configuration for fault tolerance.

• Example: 20,000sq ft. datacenter at 120w/sq ft.

• Electricity cost: $0.10/kWh

• 95% efficient vs 92% efficient UPS

• $72,000 annual difference in electricity costs

• Generated $72,000 worth of heat!!

GREEN COMPUTING

CS 595

LECTURE 12

4/1/2015

GREEN COMPUTING

• Current system extremely wasteful

• Need energy to power

• Need energy to cool

• 1000 racks, 25,000 sq ft, 10MW for computing, 5MW to dissipate heat

• Need a system more efficient, less expensive strategy with immediate impact on energy consumption

DATA CENTERS• Focus by green computing movement on data centers (SUVs of the

tech world)

• 6,000 data centers in US

• Cost: $4.5 B (more than used by all color TVs in US)

• In 2007, DOE reported data centers 1.5% of all electricity in US

• Greenhouse gas emission projected to more than double from 2007 to 2020

DATA CENTERS

• Within a few years, cost of power for data center was expected to exceed cost of original capital investment

GOAL

• Fed. Gov. wanted data center energy consumption to be reduced by at least 10%

• Same as energy consumed by 1M average US households

DATA CENTER METRICS

• Metrics

• SPECPowerjbb benchmark and DCiE from Green Grid

• Green Grid – group of IT professionals

• Power Usage Effectiveness PUE

PUE = Total facility power/IT equipment power

• Data Center infrastructure Efficiency metric DCiE

1/PUE

FUTURE VISION

• Sources of computing power in remote server warehouses

• Located near renewable energy sources – wind, solar

• Usage shifts across globe depending on where energy most abundant

CURRENT APPROACHES

• Some “low hanging fruit” approaches

• Orient racks of servers to exhaust in a uniform direction

• Higher fruit - Microsoft

• Built near hydroelectric power in WA

• Built in Ireland - can air cool, 50% more energy efficient

• Countries with favorable climates: Canada, Finland, Sweden and Switzerland

CURRENT APPROACHES

• Google – trying to reduce carbon footprint

Carbon footprint includes direct fuel use,

purchased electricity and business travel,

employee commuting, construction, server manufacturing

• According to Google, its data centers use ½ industry’s average amount of power

• How? Ultra efficient evaporative cooling (customized)

• Yahoo (are they back??)

• Data centers also carbon-neutral because of use of carbon offsets

CURRENT APPROACHES

• US government

• EPA has phase-one of Energy Star standards for servers

• Measure server power supply efficiency and energy consumption while idle

• Must also measure energy use at peak demand

• Green Grid consortium

• Dell, IBM, Sun VM-Wear AMD

• Green500 – 500 most green supercomputers

DATA CENTER PRODUCT SPECIFICATION COMPLETION• 2009

• Servers v1.0

• 2011

• Data Center Buildings Program

• 2012

• UPS v1.0 (uninterruptable power supply)

• 2013

• Servers 2.0

• Storage v1.0

• 2014

• Large Network Equipment v1.0

• 2015

• Data Center Cooling Equipment v1.0

CURRENT APPROACHES• Replace old computers with new more energy-

efficient

• But manufacturing through day-to-day uses energy

• Dell - reducing hazardous substances in computers, OptiPlex 50% more energy efficient

• Greenest computer company – VirtualBoxImages

• What is “Greenest computer ever” ?

• Is MacBook air (pro) greenest?

• https://www.apple.com/macbook-pro/environment/

GOALS FOR FUTURE

1. Consider energy to manufacture, operate, dispose of wastes

2. Sense and optimize world around us

3. Predict and respond to future events by modeling behavior (grown in performance)

4. Benefit of digital alternative to physical activities

• E-newspapers, online shopping

• Personal energy meter??

THE CASE FOR ENERGY-PROPORTIONAL COMPUTINGBARROSO AND HOLZLE (GOOGLE)

INTRO

• Energy proportional computing primary design goal for servers

• Cooling and provisioning proportional to average energy servers consume

• Energy efficiency benefits all components

• Computer energy consumption lowered if:

• Adopt high-efficiency power supplies

• Use power saving features already in equipment

INTRO

• More efficient CPUs on chips based on multiprocessing has helped

• But, higher performance means increased energy usage

SERVERS

• Servers

• Rarely completely idle

• Seldom operate at maximum

• 10-50% of max utilization levels

• 100% utilization not acceptable for meeting throughput, etc. – no slack time

SERVERS• Completely idle server waste of capital

• Difficult to idle subset of servers

• Servers need to be available

• Perform background tasks

• Move data around

• Help with crash recovery

• Applications can be restructured to create idle intervals

• Difficult, hard to maintain

• Devices with highest energy savings, highest wake-up penalty, e.g disk spin up

ENERGY EFFICIENCY AT VARYING UTILIZATION LEVELS

• Utilization – measure of performance normalized to performance at peak loads

• Energy efficient server still consumes ½ power when doing almost no work

• Power efficiency – utilization/power value

• Peak energy efficiency occurs at peak utilization and drops as util. decreases

• At 20-30% utilization, efficiency drops to less than ½ at peak performance

TOWARD ENERGY-PROPORTIONAL MACHINES

• Mismatch between servers’ high-energy efficiency characteristics and behavior

• Designers need to address this

• Design machines that consume energy in proportion to amount of work performed

• No power when idle (easy)

• Nearly no power when little work (harder)

• More as activity increases (even harder)

CPU POWER• Fraction of total server power consumed by CPU changed since

2005

• CPU no longer dominates power at peak usage, trend will continue

• Even less when idle

• Processors close to energy-proportional

• Consume < 1/3 power at low activity (70% of peak)

• Power range less for other components

• < 50% for DRAM, 25% for disk drives, 15% for network switches

DYNAMIC RANGE• Processors can run at lower voltage frequency mode without

impacting performance

• No other components with such modes• Only inactive modes in DRAM and disks

• Inactive to active mode transition penalty (even it only idle to submilliseconds)

• Servers with 90% dynamic range could cut energy by ½ in data centers

• Lower peak power by 30%

• Energy proportional hardware reduce need for power management software

DISKS - INACTIVE/ACTIVE• Penalty for transition to active from inactive state makes it less

useful

• Disk penalty 1000 higher for spin up than regular access latency

• Only useful if idle for several minutes (rarely occurs)

• More beneficial to have smaller penalty even if higher energy levels

• Active energy savings schemes are useful even if higher penalty to transition because in low energy mode for longer periods

CONCLUSIONS

• CPUS already exhibit energy proportional profiles, other components less so

• Need significant improvements in memory and disk subsystems

• Such systems responsible for larger fraction of energy usage

• Need energy efficient benchmark developers to report measurements at nonpeak levels for complete picture

GREEN INTROSPECTION BY K. CAMERON

HISTORY OF GREEN

• In the 1970s

• Energy crisis

• High gas prices

• Fuel shortages

• Pollution

• Education and action

• Environmental activism

• Energy awareness and conservation

• Technological innovation

GIFTS FROM THE 70S

• Energy crisis subsided

• In the meantime advances in computing responsible for:

• Innovation for energy-efficient buildings and cars

• Identified causes and effects of global climate change

• Grassroots activism, distributing info about energy consumption, carbon emission, etc.

• The same computing technologies pioneered by hippie geeks (???) are the problem now

WHAT HAPPENED NEXT

• Call to action within IT community (what about the 80s??)

• In 1990s

• General-purpose microprocessors built for performance

• Competing processors

• ever-increasing clock rates and transistor densities

• fast processing power and exponentially increasing power consumption

• Power wall at 130 watts

• Power is a design constraint

BETTER, BUT ALSO WORSE?

• To reduce power consumption

• Multicore architectures – higher performance, lower power budgets

• But

• Users expect performance doubling every 2 years

• Developers must harness parallelism of multicore architectures

• Power problems ubiquitous – energy-aware design needed at all levels

MORE PROBLEMS

• Memory architectures consume significant amounts of power

• Need energy-aware design at systems level

• Disks, boards, fans switches, peripherals

• Maintain quality of computing devices, decrease environmental footprint

• Can’t rely on nonrenewable resources or toxic ingredients

THOSE DATA CENTERS

• IT helping in data centers

• Reducing energy with virtualization and consolidation

• Need to address chip level device to heating/cooling of building

• Need metrics

TRADE-OFF

• How often to replace aging systems?

• 2% of solid waste comes from consumer electronic components

• E-waste fastest growing component of waste stream

• In US 130,000 computers thrown away daily and 100 M cell phones annually

• Recycle e-waste (good luck)

• Use computers as long as possible?