Dialing it in: A Year of Tuning the NCAR-Wyoming Supercomputing Center

Dialing it in: A Year of Tuning the NCAR-Wyoming Supercomputing Center

Aaron Andersen, Deputy Director of Operations, National Center for Atmospheric Research(NCAR)

Aaron AndersenNov., 2013

“… to provide, or arrange for provision of facilities for the scientific community as a whole whose initial cost and upkeep lie beyond the capability of individual universities or research groups.”

Preliminary Plans for a National Institute for Atmospheric Research,

1959, NCAR Blue Book

NCAR’s Founding Mission


History of Computing at NCAR

Exotic CoolingFreon R-22

Fluorinert

Water

Return of Liquid CoolingWater

Warm Water

FR3


• Key to Optimum Performance (Three M’s)• Monitoring

• Measurement

• Maintenance

• Infrastructure Support Group – Cheyenne• Electricians

• Mechanics

• Building Automation Controls

• HVAC Technician

• Commissioning Agent• Measurement & Verification 12 months post completion


Tuning the Facility – Student Interns

• Clair Christofersen - 2012• University of Miami Ohio

• BS Mechanical Engineering – Applied Math

• Air Flow Modeling (CFD Analysis)

• Jason Jones - 2012• Clemson University -> Georgia Tech

• BS Mechanical Engineering

• Air Flow, Water Flow and Total Energy

• Ademola Olarinde - 2013• Texas A&M Kingsville

• MS Mechanical Engineering


65F (18.3 C) Degree Chilled Water

Evaporative Solution


Liquid Cooling – ASHRAE 2011 Guidelines

121 hrs1.38%

8639 hrs98.62%


Characterize Building Response


• Saw an increase in mechanical consumption in winter months

• Energy Recovery Heat Pumps consuming way more electricity than expected

• Why do we care?• $5000 per month

• What is the root cause?

Mechanical Load in Winter


• Heat Rejection from Yellowstone is too low 1.2MW vs. 4MW Design

• Significantly greater heating demand from the building during winter months

• Commissioning Agent identified the same problem and possible culprit, large Air Handling Units

Root Cause


Why This Matters


• UPS Efficiency• Only Critical Systems are Covered on UPS

• Disk• Network• AuthN/AuthZ• Login• Management

• Low load causes low efficiency

• Firmware upgrade moves UPS efficiency from 70% ~ 95%

• Maintenance really matters• Cleaning strainers improve pump energy by 20 KW

• More frequent maintenance than recommended

• Water Usage• Zero Blow down for Cooling Tower saving 40%

• Negotiated reduction in Sewer cost ~$2500/month

Other Issues


PUE Trend Since Startup


PUE – One Year Aug 2012 – July 2013

Documents

Dialing it in: A Year of Tuning the NCAR-Wyoming Supercomputing Center