Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
APC by Schneider Electric – July 2017
Best practice and Infrastructure
Guideline FOR On-Premise Data Center
Patphong Petkaewna
DC Solution Architect - International
APC by Schneider Electric – Patphong – July 2017
Explosive Growth
●… driven by the number of connected users,
& the Internet of Things (IoT)
21 Billion network
devices
TB transmitted
(flight hr.)
2
1.3Million video views
(per min.)
40
Billion Internet users
APC by Schneider Electric – Patphong – July 2017
Trends
Energy Efficiency
(Do more with less)
Everything as a Service
Speed & Simplification
Internet of Things
Capital Preservation
Market
Social Technologies
Big Data
Mobile Computing
Edge Compute
Converged IT
Cloud Computing
Security
Outsourcing
Medium Voltage
Distribution
New cooling techniques
Servers w/ UPS
Prefabrication
Micro Data Centers
Lower Tier Designs
Relaxed Operation
Standards
IT Physical Infrastructure
APC by Schneider Electric – Patphong – July 2017
Unprecedented Changes
Controlling Energy and Service Cost
BalancingAvailability through Hardware and Software
Leveraging Prefabrication
Integrating On-premise with Outsourced IT
Introducing Edge Computingfor Reduced Latency
Achieving Higher Energy Efficiency
Meeting Regulatory Requirements
Leveraging Software to Automate Services
Deploying Converged Infrastructure
Designing for Unknown,Future IT
Managing IT and Facility Assets
Evaluating or Adopting Open Compute Solutions
Operating a Bimodal IT Environment that is both Stable and Agile
APC by Schneider Electric – Patphong – July 2017
Increasingly Complex JobIn the future, 60% of a Data Center’s assets will be cloud or collocation based.
CUSTOMERS,
DATA, &
CRITICALITYColocation
Cloud
Services
Cloud
Services
Edge
On Premise
data
Leads to the NEED for more:
OPTIMIZATION & INSIGHT,
plus MEGA DATA CENTERS &
COLOCATION FACILITIES
More
IDC FutureScape: Worldwide
Datacenter 2015 Predictions
APC by Schneider Electric – Patphong – July 2017
How will distribution change?
●By 2025, hyper-scale Data Centers will
contain 1/3 of the world’s Processing Power
Retrofitting(or investing in)
existing DCBuilding Outsourcing
Enterprises
& Government
Cloud
& ColosSMB
APC by Schneider Electric – Patphong – July 2017
Power to
data centerCooling
Lights
Fire
Security
Generator
Switchgear
Physical
infrastructure
IT Load
Power
to IT
UPS
PDU
Cabling
Switches
Etc.
Power path
to IT
Power to
other support
USEFUL
Power= Power consumed
by IT loads
SUPPORT
Power= Power consumed
by support
equipment
Losses in
power system
Goal is to
reduce this
Data Center Power Consumption
In many data
centers, less than
half the electricity
makes it to the
computer loads.
APC by Schneider Electric – Patphong – July 2017
Where Does the Power Go?
Power flow in a typical 2N
data center at 50% load
Power enters the
data center as
electrical energy
Virtually all power
(99.99%+) leaves the
data center as heat.
The rest is converted by IT equipment.
PUE = estimated annual PUE
APC by Schneider Electric – Patphong – July 2017
Data Center Applications Affect Efficiency
Increased Efficiency
High density or dynamic loading
applications offer an opportunity for
increased efficiency (if supported by
smart row-based power and cooling).
Reduced Efficiency
Reduced operating loads and
over-capacity of power and
cooling reduces efficiency.
APC by Schneider Electric – Patphong – July 2017
Point (Computer Equipment) Solutions
Virtualization
Has the potential to reduce the total
amount of IT equipment required to
perform a specific function.
This offers a means to reduce IT-
load power consumption.
IT systems operating at higher
densities with time-varying
power draw are driving down
electrical efficiency.
APC by Schneider Electric – Patphong – July 2017
Best Practices for Low PUE
“If you pick the best-of-breed commercial products and you combine that
with best practices in terms of how you design and run your operation,
you’re looking at an overhead of about .24.”
~ Chris Malone, thermal technologies architect for Google
Fix cooling first
● Close-coupled cooling
● Separate hot and cold aisles
● Efficient cooling-plant designs
● Tame energy consumption
● Heat exchanger in line with chiller
http://www.datacenterdynamics.com/focus/archive/2011/05/google-low-pue-is-easy-if-you-follow-best-practices
Optimize power distribution
and backup
● Minimize the number of
conversions
● Use efficient UPS units
APC by Schneider Electric – Patphong – July 2017
Scalable power and cooling
Row-based cooling
High-efficiency UPS
AC power distribution
Variable-speed drives on pumps and chillers
Proposals to Address Waste of Power
APC by Schneider Electric – Patphong – July 2017
Five Key Contributors to Inefficiency
These all contribute to SUPPORT efficiency.
1 Inefficiencies of the power equipment
2 Inefficiencies of the cooling equipment
3 Power consumption of lighting
4 Over-sizing of the power and cooling systems
5 Inefficiencies due to configuration
Each of these are analyzed in detail in APC White Paper 113 – Electrical Efficiency Modeling for Data Centers,
APC by Schneider Electric – Patphong – July 2017
1.) Inefficiencies of Power Equipment
When the equipment is doubled for redundancy or operated
well below its rated power, efficiency falls dramatically.
Efficiency values are misleading and cannot be used
to calculate the power wasted in real installations
APC by Schneider Electric – Patphong – July 2017
2.) Inefficiencies of Cooling Equipment
The inefficiency
(waste heat) of cooling
equipment typically
greatly exceeds the
inefficiency of power
equipment.
An increase in the
efficiency of the
cooling equipment
directly benefits overall
system efficiency.
APC by Schneider Electric – Patphong – July 2017
The Goal of Cooling Distribution
To create clear paths for
cooled air and hot exhaust
air
Need to reduce or
eliminate the cold and hot
air leakage
CRAH
Front
IT
Rack
Front
IT
Rack
Front
IT
Rack
Cold air
leakage
Hot air
leakage
APC by Schneider Electric – Patphong – July 2017
BYPASS AIRFLOW
● A comprehensive study
performed by Upsite
Technologies Inc. concluded
that 60% of the air supplied in
traditional data centers is
wasted due to bypass airflow
● Significant efficiencies and cost
savings can be realized by
eliminating bypass airflow.
APC by Schneider Electric – Patphong – July 2017
RECIRCULATION
● Hot air exhaust circulating back into its own intake can cause device thermal overload.
● Typical manufacturer inlet temperature threshold for device operations is 35°C.
● Exceeding manufacturers operating device threshold can lead to unplanned computing system outages and data loss.
APC by Schneider Electric – Patphong – July 2017
TEMPERATURE STRATIFICATION
● Significant gradient of air temperatures beyond ASHRAE TC9.9 places devices at risk of thermal overload
● Maintaining inlet temperature gradients within the ASHRAE recommended range significantly saves energy
APC by Schneider Electric – Patphong – July 2017
The Goal of Cooling Distribution
To create clear paths for
cooled air and hot exhaust
air
Need to reduce or
eliminate the cold and hot
air leakage
CRAH
Front
IT
Rack
Front
IT
Rack
Front
IT
Rack
Cold air
leakage
Hot air
leakage
Need to design for Thermal
ride through.
APC by Schneider Electric – Patphong – July 2017
Thermal run-away challenge in high density
source: WP 105, Data Center Thermal Run away
There is a compelling reason for exploring chilled water cooling for High density
data centers.
APC by Schneider Electric – Patphong – July 2017
3.) Power Consumption of Lighting
The heat generated by lighting
must be cooled by the cooling
system, which causes the air
conditioning system to
consume correspondingly
more electrical power.
Increases in the efficiency of the lighting, or controlling lighting to be present only
when and where needed, materially benefits overall system efficiency.
APC by Schneider Electric – Patphong – July 2017
4.) Over-sizing
Over-sizing occurs when the
design value of the power
and cooling system exceeds
the IT load.
The average data center is
ultimately oversized by three
times in design value.
Capital cost of power/cooling systems = $5 per Watt
● 70% is wasted
Excess electricity costs are significant when data centers or network
rooms are oversized.
● Idling loss of power system + cooling costs = 10% of power rating
See APC White Paper 113, Electrical Efficiency Modeling for Data Centers.
APC by Schneider Electric – Patphong – July 2017
5.) Inefficiencies Due to Configuration
The physical configuration of the IT equipment can have a dramatic effect
on the energy consumption of the cooling system. Poor configurations:
● Force the cooling system to
move more air than required.
● Cause the cooling system to
generate cooler air than
required.
● Force cooling units into a
conflict where one is
dehumidifying while another
is humidifying.
These configuration problems are present in virtually all operating
data centers today and cause needless energy waste.
APC by Schneider Electric – Patphong – July 2017
Common Ineffective Hot Spot Solutions
● Decrease the cooling system set points
● Add more cooling units
● Place large circulating fans in front of the rack
APC by Schneider Electric – Patphong – July 2017
Improved Data Center Architecture –
Available and Practical Today!
A “system” makeover that combines the best elements of data center design:
● Engineering design of
individual devices
● Power distribution
● Inter-component
communication and
coordination
● Cooling strategy
● System planning
● Management tools
When all of these elements are combined as an integrated system, the performance improvement can be dramatic.
APC by Schneider Electric – Patphong – July 2017
Data Center Design input…
● Uptime Institute – TIER LEVEL
● PUE
●ASHRAE – TC 9.9
● Location, Density, temperature, redundancy, water
availability, building constraints
APC by Schneider Electric – Patphong – July 2017
Tier Classifications
APC by Schneider Electric – Patphong – July 2017
Standard / Guideline for Data Center Design area
Page 29Confidential Property of Schneider
Electric |
Tier I
Tier II
Tier III
Tier iV
APC by Schneider Electric – Patphong – July 2017
Power Usage Effectiveness- PUE
APC by Schneider Electric – Patphong – July 2017
ASHRAE TC 9.9 Requirements
Temperature Range:
• in the suction section of the rack (cold aisle)
the air temperature must be at a temperature
of between 18°C and 27°C;
•hot aisles situated at the discharge section of
the rack have a temperature of between 30
and 38°C depending on the active thermal
load.
APC by Schneider Electric – Patphong – July 2017
ASHRAE TC 9.9 Requirements
Humidity Range:
•Humidity should be less than 60% with
the lower and upper dew point
temperatures of 5.5°C and 15°C,
increasing the traditional allowed range
for humidity control.
APC by Schneider Electric – Patphong – July 2017
Optimize the Architecture
● Turn it off!
● Reduce over-sizing
● Take advantage of technology
● Pay close attention to rated capacity
● Beware of stranded capacity
Data center efficiencies can be substantially improved when an integrated system is developed based on the following principles:
APC by Schneider Electric – Patphong – July 2017
Optimize the Physical ConfigurationOptimized, integrated physical configuration should be inherent within the
system, and not tied to the characteristics of the room where it resides.
Configuration should include:
● Integrated row-based cooling
● Instrumentation to identify and warn about conditions that generate sub-optimal
electrical consumption
● Installation and operation tools that maximize operating efficiency
APC by Schneider Electric – Patphong – July 2017
Commercially Available Integrated DC How does this address inefficient design factors?
Batteries Scalable
UPS
Row-based
CRAC
High-efficiency
transformerless PDU
● Power equipment
● Cooling equipment
● Lighting
● Over-sizing
● Configuration
APC by Schneider Electric – Patphong – July 2017
Cost Savings by Subsystem
APC by Schneider Electric – Patphong – July 2017
$0 $200,000 $400,000 $600,000 $800,000 $1,000,000
UPS
PDU
Generator
Switchgear
Distribution Wiring
CRAC
Heat Rejection
Pumps
Chiller
Humidifier
Lights
Aux Devices
IT Load
$ per Year Electric Cost @ $.10/kwhr
Improved Architecture
Traditional Baseline
In terms of PUE efficiency value*:
Traditional data center = PUE of ~ 2.5
Improved architecture = PUE ~ 1.5
*Operating at 50% of rated IT load, under
the same conditions.
Eliminating dual path power redundancy or N+1 air handlers
would cause efficiencies to rise and savings to fall somewhat.
Images available at: http://www.apc.com/tt?tt=6
Traditional Improved
APC by Schneider Electric – Patphong – July 2017
How Does Improved Architecture
Achieve Savings?
The key elements that give rise to the improvements of the
new architecture include:
1. Scalable power and cooling
● To avoid over-sizing
2. Row-based cooling
● To improve cooling efficiency
● Free Cooling also provides
regionally derived cost savings
3. High-efficiency UPS
● To improve power efficiency
APC by Schneider Electric – Patphong – July 2017
How Does Improved Architecture
Achieve Savings?
4. 415/240 V AC power distribution
● To improve power efficiency
5. Variable-speed drives on pumps/chillers
● To improve efficiency at partial load and on
cool days
6. Capacity management tools
● To improve utilization of power, cooling,
and rack capacity
7. Room layout tools
● To optimize layout for cooling efficiency
The key elements that give rise to the
improvements of the new architecture include:
APC by Schneider Electric – Patphong – July 2017
Data center efficiency as a function of IT load comparing modular vs.non-modular designs
Scalable Power and Cooling
At lower IT loads the
efficiency always
declines and is equal
to zero when there is
no IT load.
To correct the problem of reduced efficiency due to an over-sizing condition,
the power and cooling equipment could be scaled over time to meet the IT
load requirement.
Efficiency Varies with IT load
APC by Schneider Electric – Patphong – July 2017
Scaled Buildout: PUE and TCO Advantage
Savings from
scaled deployment
Capex 38%
Opex 19%
Non-energy 44%opex
TCO 33%savings
TCO
Right-sizing benefits over a 10-year data center lifetime
SCALABLE
ADAPTABLE
4.0
3.5
3.0
2.5
1.5
2.0
UPFRONT buildout – day one
SCALED buildout – as needed
1 3 4 5 6 7 8 9 102
PUE
Data center lifetime (years)
See APC White Paper 143, Data Center Projects: Growth Model
APC by Schneider Electric – Patphong – July 2017
Row-based Cooling to Improve
Cooling Efficiency
In-row air
conditioner
Hot aisle air enters from rear,
preventing mixingCold air is supplied to
the cold aisle
Heat captured and
rejected to chilled water
Cold Aisle
Cold Aisle
Row-based cooling is a key enabler of cost-
effective implementation of room layout and
capacity management tools, and scalable cooling
APC by Schneider Electric – Patphong – July 2017
High-efficiency UPS to Improve Power
Efficiency
LBNL report on UPS efficiency: http://hightech.lbl.gov/documents/UPS/Final_UPS_Report.pdf, Figure 17, page 23.
Efficiency gain is
greatest at
lighter loads.
APC by Schneider Electric – Patphong – July 2017
415/240V AC Power Distribution to
Improve Power Efficiency
Eliminates power distribution unit (PDU) transformers and their associated losses.
Efficiency reduction of transformer-based PDUs = 2% to 15%
Larger percent losses occur in data centers operating with redundant power paths and lighter IT loads.
208 V
Neutral
Typical
Load
Typical
Load
Neutral
240 V
Adoption of the European standard of 415/240V offers significant efficiency improvement for North America Data Centers.
See APC White Paper 128, Increasing Data Center Efficiency by Using Improved High-Density Power Distribution.
APC by Schneider Electric – Patphong – July 2017
Variable-speed Drives on Pumps and
Chillers to Improve Cooling EfficiencyChillers and pumps with fixed-speed motors are configured for maximum expected
load and worst case (hot) conditions, and therefore spend much of their operating
time with their motors working harder than necessary.
Pumps and chillers equipped with variable-speed drives (VFDs) and
appropriate controls can reduce their speed and energy consumption to match
the current IT load and outdoor conditions. The energy improvement varies, but
can be as large as 10% or more.
APC by Schneider Electric – Patphong – July 2017
Capacity Management Tools Improve
Utilization of Power, Cooling, and Rack
Capacity
● Most data centers do not fully utilize power, cooling, and rack capacity
● Modern IT equipment = 5 to 20 kW per cabinet
● Typical data center operation ≤ 3 kW per cabinet
●Efficiency gains:
● Shorter airflow paths = less fan horsepower
● Less air mixing = higher heat rejection temperatures = improved chiller
efficiency & increased CRAC capacity
● Shorter wiring lengths = less wiring and PDU losses
See APC White Paper 150, Power and Cooling Capacity Management for Data Centers.
More IT load can be powered by the same
power and cooling infrastructure
APC by Schneider Electric – Patphong – July 2017
Capacity Management Tools
5% in overall infrastructure electrical efficiency
5-10% capital savings due to higher power density
Drill in to row
or rack level
Graphic representation of
data center floor layout
Visibility to average
and peak power
usage by measuring
actual consumption
APC by Schneider Electric – Patphong – July 2017
Room Layout Tools Optimize Layout for
Cooling Efficiency
● Minimized airflow path
lengths (reduce fan power)
● Minimized airflow resistance
(reduce fan power)
● Maximized heat transfer - IT
equipment exhaust air returned
directly to the air conditioner
● Balanced airflow capacities
to nearby load airflow
requirements
Room layout optimized for efficiency =
APC by Schneider Electric – Patphong – July 2017
Overall Infrastructure Efficiency Gains
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
% IT Load
% E
ffic
ien
cy
Redundant (Dual power path, N+1 air handlers)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
% IT Load
% E
ffic
ien
cy
Non-Redundant (Single power path, N air handlers)
Traditional data centers:
• Power and cooling redundancy reduces the overall efficiency by about 5%
Improved architecture:
• Power and cooling redundancy have a negligible effect on efficiency
Phased deployment of modular power and cooling has the
most significant efficiency benefit for data centers
APC by Schneider Electric – Patphong – July 2017
Other Factors that Affect Efficiency Gains
Helpful Factors
● Dropped ceiling for air return
● Full dual-path air handlers
● Full dual-path chillers
● Energy-efficient lighting
● Powering air handlers from the UPS
● Packaged chillers or DX glycol systems
Hindering Factors
● Uncoordinated room-perimeter air conditioners
● Lack of hot-aisle/cold-aisle rack layout
● Imbalanced sizing of power and cooling systems
● Shallow raised floor (0.5 m or less)
● Large auxiliary loads
● Hot and/or humid climate
● Very long coolant pipe runs
APC by Schneider Electric – Patphong – July 2017
Data Center Efficiency Calculator:
TradeOff Tools
http://tools.apc.com
● High-level planning tools
● Show actual implications of various deployment decisions
● Accurately calculate potential impacts on a data center
APC by Schneider Electric – Patphong – July 2017
Conclusion
● There is pressure to reduce energy consumption of data centers
● Traditional power and cooling systems are inefficient, stranded
capacity rates are high, and server utilization is low
● The users of energy have an equally important role in improving
electrical efficiency
● Data centers can begin by ”rightsizing” their power and cooling
infrastructure so that the actual IT load operates closer to full load
capacity
Why a more efficient architecture?