View
7
Download
0
Category
Preview:
Citation preview
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Using CFD Analysis to Predict Cooling System Performance in Data Centers
Ben Steinberg, P.E.Staff Applications Engineer
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Agenda
What is CFD?Applications of CFDApplicability to the Data CenterHow it worksExamples
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
What is CFD?
CFD stands for Computational Fluid DynamicsFluid Dynamics is the study of the motion of fluids (liquids and gases)Complex equations that describe fluid flow and heat transfer were developed hundreds of years agoFor simple systems, these equations can be solved manually
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Applications of CFDComputers make solving these equations possible for very complex systems.Developed in the 1960’s for aerospace, defense and nuclear power industries.Usage grew into the automotive industry in 1970’s.CFD is used as a “virtual wind tunnel” for designing and testing airplanes, cars and ships before a prototype is even built.Even used by hospitals to simulate flows of fluids through the human body.
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Applicability to the Data CenterIn 1965, Intel co-founder Gordon Moore predicted that the number of transistors on a chip would double about every 2 years.A musical birthday card costing a few U.S. dollars today has more computing power than the fastest mainframes of a few decades ago.A single rack of servers can now consume as much power and create as much heat as 3 household ovens!
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Increasing Heat Densities
05
101520253035
'95 '96 '97 '98 '99 '00 '01 '02 '03 '04 '05
Year
Per R
ack P
ower
Den
sity (
KW)
Max per-rack IT Load
Data Center rated rack power capacity
IT Loads are greatly exceeding rated capacity!
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Survey Question #1
Do you currently have hot spots in your (or a customer’s) data center?
YesNo
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Applicability to the Data Center: PastIn general, the heat load of a data center was (and still is) about equal to the power consumption of the IT equipment, in addition to other miscellaneous loads (lighting, solar, etc.)In the past, ensuring that the installed cooling capacity was at least as much as the heat load was enough to cool the IT equipment.Raised floors and overhead ducts were the most common types of air distribution.Perforated tiles in raised floors or supply grilles in overhead ducts were placed as necessary to supply cool air to the equipment inlets.
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Applicability to the Data Center: Present (and Future)
Today’s high density loads require so much cooling and airflow (up to 7 tons and 2,400 CFM per rack) that it is difficult to supply the correct quantities of cool air where it is needed (see next slide).Hot spots are created in areas where not enough cool air is supplied.Reactions to hot spots include lowering CRAC set points and adding more CRAC units, both of which can make things worse.Every data center is unique:- Size, shape, type of equipment- Changes very frequentlyVery difficult to predict performance and level of redundancy.
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Grate tilePerf tile
Blade Ser
vers
Standard IT Eq
uipment
WithEffort
TypicalCapability Extreme Impractical
12
10
8
6
4
2
00 100 200 300 400 500 600 700 800 900 1000
[47.2] [94.4] [141.6] [188.8] [236.0] [283.2] [330.4] [377.6] [424.8] [471.9]
Floor Tile Cooling Ability Requires careful raised floor design, careful CRAC placement,
and control of under-floor obstacles (pipes/wiring)300-500 cfm
RackPower(kW)
that can becooled by one tile with this
airflow
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
How it worksApplicable to existing as well as planned installations.All equipment is entered into a 3D computer model- Physical size- Power and cooling capacity- Air flowWhen the model is complete, the computer solves the equations.- could take hours or days, depending on size and detail.After solving, results can be manipulated to show temperatures, pressures and flows in different locations.Model should be maintained as equipment changes take place.
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
CFD Examples
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Raised floor cooling CFD exampleDesign Parameters:
4kW per rack / 40 racks total Hot Aisle/Cold Aisle layoutTotal power 160kWRoom size: 38' x 38' x 14' high2' raised floor w/25% perforated tiles in front of racks and PDUs5 CRAC units N+1 redundancy at room level - 4 units running at the same time = 160kWCRAC units are standard chilled water 40kW (12 ton) downflow
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Raised floor cooling CFD exampleH
OT
AIR
HO
T AI
R
COLD AIR/PERFORATED TILES
COLD AIR/PERFORATED TILES
COLD AIR/PERFORATED TILES
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Raised floor cooling CFD example
Redundancy:
Any unit failure results in loss of
cooling to an area
N+1 at room level does provide
adequate cooling in failure modes
Sectional Plane @ 5’-6” from Raised Floor
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Survey Question #2
What is your (or your customer’s) highest rack density today?
Less than 1 kW per rack1 - 2 kW per rack3 - 4 kW per rack5 - 6 kW per rackMore than 6 kW per rackNot sure
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Hard floor cooling CFD exampleDesign Parameters:4kW per rack / 40 racks total Total power 160kWRoom size: 38' x 38' x 14' highRaised floor not used for air distribution6 in-row cooling unitsN+1 redundancy at row level - 4 units running at the same time = 160kW
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
InRow Architecture
Hard floor cooling CFD example
CO
LD A
IR
HO
T AI
R
CO
LD A
IR
HO
T AI
R
CO
LD A
IR
Net
wor
kAir
IR
Net
wor
kAir
IRN
etw
orkA
ir IR
Net
wor
kAir
IR
Net
wor
kAir
IRN
etw
orkA
ir IR
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Sectional Plane @ 5’-6” from Datacenter Floor
InRow Architecture
Hard floor cooling CFD example
Redundancy:
Normal Operating
Mode 4 units running
No Hot Spots
developed in front of
IT equipment.
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Raised floor cooling CFD exampleDesign Parameters:
2kW per rackNot Hot Aisle/Cold Aisle layoutTotal power 138kWRoom size: 53' x 41' x 9' high2' raised floor w/25% perforated tiles in front of racks4 CRAC units N+1 redundancy at room level - 3 units running at the same time = 150kWCRAC units are glycol cooled DX 50kW (15 ton) downflow
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Temperature DistributionTop View, top half of racks
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Temperature DistributionSide View
•Cool air from floor plenum is consumed by bottom half of racks; top half is left to ingest hot air from recirculation.
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Temperature DistributionTop View, top half of racks
•New CRAC unit is added; note the cooler temperatures around the room.
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Under floor pressure & flow CFD example
1 foot raised floor height2kW per rackAnalysis of 2 different CRAC layouts
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Temperatures
Scenario 1 Scenario 2
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Plenum Airflow
Raised floor hidden for clarity.
Scenario 1 Scenario 2
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Plenum Pressure
Raised floor and IT racks hidden for clarity.Note dead spots at center of recirculations.
Scenario 1 Scenario 2
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Perforated Tile Flow Rates
flow rate per tile (cfm)200 425 650 875 1100
speed (ft/min)•Very Non-uniform distribution
•BACKFLOW here
Scenario 1 Scenario 2
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Conclusions
As equipment power densities rise, so does the amount of heat produced.Very difficult to accurately predict performance and redundancy at high densities.Recent advances in computer power and software have made CFD a viable tool to help fix or prevent hot spots from occurring in the data center.
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Using CFD Analysis to Predict Cooling System Performance in Data Centers
QUESTIONS
© 2006 American Power Conversion Corporation. All rights reserved. All trademarks provided are the property of their respective owners.
Using CFD Analysis to Predict Cooling System Performance in Data CentersAgendaWhat is CFD?Applications of CFDApplicability to the Data CenterIncreasing Heat DensitiesSurvey Question #1Applicability to the Data Center: PastApplicability to the Data Center: Present (and Future)How it worksCFD ExamplesRaised floor cooling CFD exampleSurvey Question #2Hard floor cooling CFD exampleRaised floor cooling CFD exampleTemperature DistributionTop View, top half of racksTemperature DistributionSide ViewTemperature DistributionTop View, top half of racksUnder floor pressure & flow CFD exampleConclusions
Recommended