Modular Data Center Design#CSEmodulardatacenter

Sponsored by:

Today’s Webcast Sponsors:

Learning objectives

•Learn the differences between the various types of modular data centers.•Know about the benefits—and negative aspects—of specifying modular data centers.•Understand the cooling requirements and issues associated with modular data centers, including compliance with ASHRAE Standard 90.1 and the International Energy Conservation Code (IECC).•Understand the unique power/electrical requirements and issues associated with modular data centers.

Definitions

MDF: main distribution frame

IDF: intermediate distribution frame

ISO: International Organization for Standardization

PUE: power usage effectiveness

UPS: uninterruptible power supply

#CSEmodulardatacenter

Presenters:

Bill Kosik, PE, CEM, BEMP, LEED AP BD+CHP Critical FacilitiesData Center Energy Technologist

Brian Rener, PE, LEED APM+W U.S. Inc. - A Company of the M+W GroupElectrical Engineering DisciplinePlatform Leader Quality Assurance Manager

Amara Rozgus

CFE Media

Consulting-Specifying Engineer and Pure Power

Editor in Chief/Content Manager

Sponsored by:

Modular Data Center Design

#CSEmodulardatacenter

Aiming for the data center of the future

• Efficient and effective

• Self-regulating

• Standardized processes

• Fully available and resilient

• Monitoring & control

• Fully service-oriented

• Green

• Business-centric • Integrated

• Modular and elastic

• Shared resource pools

• Policy-based

• Fully automated

• Right sourcing

(1) IDC Directions 2013: Why the Datacenter of the Future Will Leverage a Converged Infrastructure, March 2013, Matt Eastwood ; (2) & (3) IDC Predictions 2012: Competing for 2020, Document 231720, December 2011, Frank Gens; (4) http://en.wikipedia.org

The growing Internet of Things (IoT)

Pervasive connectivity

Explosion of information

Today

400,710 ad requests

2000 lyrics playedon Tunewiki

1,500 pingssent on PingMe

208,333 minutesAngry Birds played

23,148 apps downloaded

416,340 tweets

Smart device

expansion

60

sec

2013

30Billion

By 2020

40 Trillion GB

… for 8

Billion

10Million

DATA

(1)

(2)

(3)

Devices

Mobile Apps

(4)

The IoT is a world where nearly everything is connected to a data center, items like cars, home appliances, glasses, watches, jewelry, clothes…. even packaged goods. Every one of these devices will, one way or another, be connected to a data center for control, management, and analysis. The required data center capacity cannot be served effectively with current data center and server architectures.

Software Defined Server45 hot-plug cartridges

Compute, Storage, or Combination

• Single-server = 45 servers per chassis

• Quad-server =180 servers per chassis

Approximate average 55 W per cartridge (20 W min, 90 W max)

45 servers per chassis = 450 servers per rack = 24 kW per rack

180 servers per chassis = 1800 servers per rack = 97 kW per rack

Data centers can put up to 1800 servers in a single, 47U rack, which could take 10X as many racks using a standard architecture. This extreme density reduces, per a given unit of work, the datacenter size, energy consumption, complexity and cost.

Modular Data Center Basics• Two basic types of pre-manufactured spaces

– ISO containers– Non ISO containers– Modular rooms

Modular Data Center Basics Containerized

– Lower Capex, scalable, relocatable– ISO

• Conform to ISO standards for size. 10-, 20-, 40-, and 53-ft lengths standard, 9.5-ft width typically

• Usually built to UL standards and not occupied • Up to 19 conventional IT racks • 3 kW to 40 kW per rack and higher

– Non ISO• Can be any size • Maybe built to IFB/IFC codes • Can be occupied.

Modular Data Center Basics Modular rooms

– Prefabricated rooms, assembled on-site– Expandable construction– Rapid deployment over stick built– General same features as conventional data centers.

Modular Data Center Basics Containerized

– Many types and configurations• All in ones • IT/data only• MDF/IDFs• Power gear and UPSs• Cooling modules

– Can be located outside or inside a structure– Code officials often unfamiliar– Rapid and scalable deployment environments.

Server Power Use Efficiency

Based on testing data, the average power has been steady with an increase in 2011. The idle power as a percentage of full power has been trending downward over the testing period of 2007-2013.

Server Power and Inlet TemperatureIn general, server power demand increases commensurate with inlet temperature

This graph shows server airflow and power requirements based on inlet temperature and workload percent. Notice that the power and airflow both increase as the inlet temperature increases above 28°C, even at an idle workload.

Increasing Temperature to Reduce Energy Use

Conclusion: Using hotter inlet temperatures works well in hot climates and when using economizer. In cold climates, there is relatively little difference because economization using colder temperatures is available most of the year.

PUE Varies Based on Climate

0.35 difference in PUE based on climate and cooling system type

Modular Data Center Cooling TechnologiesDirect OA / Evaporative

Consists of a supply fan, filters, direct evaporative media and direct expansion cooling assembly. Is most efficient in cold to moderate temperature environments with low to moderate humidity levels.

Indirect Evaporative

Consists of a supply fan, filters, indirect evaporative media and direct expansion cooling assembly. Provides separation between environments with high levels of air pollution due to 100% recirculation allowing the unit to a run a closed air circuit.

Heat Wheel

Consists of multiple supply and exhaust fans, filters, heat transfer wheel and direct expansion cooling assembly. Provides isolation of the outdoor air streams where direct use in the data center is not possible.

Air-to-Air HX/Heat Pipe

Consists of multiple supply and exhaust fans, filters, heat transfer wheel and direct expansion cooling assembly. Provides isolation of the outdoor air streams where direct use in the data center is not possible.

Cooling Modularity and Energy Efficiency• Multiple cooling methods

– Adaptable to different climate zones

– Tuned to local environment

– Provide highest efficiency for a

particular location

– Use external cooling in most climates,

reducing power and water

consumption

• External containers

– Easier installation, maintenance,

upgrade

– Protection for critical IT equipment

– Add containers to conform to

increased IT loads

Modularity and Energy Efficiency

Types of Modular Data Centers Better described as Flexible Facilities

Traditional Modular Container Modular Industrialized Modular

Growth and Flexibility

Traditional Modular Industrialized

Data center

Power/cooling

Expansion

Traditional Modular Data Center

Phase 1 Phase 2 Phase 3

Power/Cooling Infrastructure

Data Center

Admin and

Support Spaces

Container Modular Data Center

DX Modules

Industrialized Modular Data Center

About the Mechanical and Electrical Systems

Summary of Optimization Levels • Extreme regional variations in CO2 from electricity generation

• Determine appropriate balance of water and electricity usage

• Climate WILL impact HVAC energy use – select sites carefully

• Use evaporative cooling where appropriate

• Economizer strategy will be driven from climate characteristics

• Design power and cooling modularity to match IT growth

• Plan for power-aware computing equipment

• Use aisle containment or direct-cooled cabinets

• Design in ability to monitor and optimize PUE in real time

• Push for highest supply temperatures and lowest moisture levels

• Identify tipping point of server fan energy/inlet temperature

• Minimize data center footprint by using high-density architecture

Dat

a C

ente

rC

limat

eS

yner

gie

sC

on

verg

ence

Modular Data Centers - ElectricalContainerized power criteria

– Typical maximum IT load is around 1 MW– Voltage levels 120/208, 400, 480, 600 Volts– Multiple power sources.

Modular Data Centers - ElectricalTier ratings

– Available from Tier I to Tier IV

Modular Data Centers - ElectricalContainer examples

– Combined DC module– Power modules

Modular Data Centers - Electrical Container layout example

Modular Data Centers - Electrical Connecting power to a container

– Hard-wired– Plug

Modular Data Centers - ElectricalPower sources

– Main switchgear/switchboards– UPS– Generators

Separate or containerized

Modular Data Centers - ElectricalPower sources

– Main gear

Modular Data Centers - ElectricalGrounding

Resources

• Consulting-Specifying Engineer• Critical Facilities Summit• Pure Power• Uptime Institute• 7x24 Exchange

#CSEmodulardatacenter

Bill Kosik, PE, CEM, BEMP, LEED AP BD+CHP Critical FacilitiesData Center Energy Technologistwjk@hp.com

Brian Rener, PE, LEED APM+W U.S. Inc. - A Company of the M+W GroupElectrical Engineering DisciplinePlatform Leader Quality Assurance ManagerBrian.Rener@mwgroup.net

Thank You!

Presenters:

Bill Kosik, PE, CEM, BEMP, LEED AP BD+CHP Critical FacilitiesData Center Energy Technologist

Brian Rener, PE, LEED APM+W U.S. Inc. - A Company of the M+W GroupElectrical Engineering DisciplinePlatform Leader Quality Assurance Manager

Amara Rozgus

CFE Media

Consulting-Specifying Engineer and Pure Power

Editor in Chief/Content Manager

Thanks to Today’s Webcast Sponsors:

Modular Data Center Design#CSEmodulardatacenter

Sponsored by: