University of Leeds, Iceotope and 3M named for innovative liquid cooled servers that cut cooling costs by up to 97%!

Case Study SnapshotA large user of High Performance Computing (HPC) for its research, the University of Leeds is currently listed in the Top500 Supercomputer Sites. The deployment of the Leeds’ Iceotope Solution augments the University’s current HPC facilities.

Funded partially by the University’s Digital Technologies Innovation Hub, the system has been installed in a large thermofluid mechanics laboratory within the School of Mechanical Engineering, where it is used to perform complex thermal Computational Fluid Dynamics (CFD) simulations in addition to becoming an experimental apparatus for exploring the possibilities of re-use of waste heat derived from Information Technology (IT) systems.

While most computer servers use air (namely fans) to cool their electronics, all of the components in the Iceotope Solution are completely immersed in 3M™ Novec™ Engineered Fluid.

Power-hungry fans of traditional computing are replaced by a silent liquid encapsulation cooling process that relies on the natural convection of heat. Iceotope’s cooling process offers incredible efficiency gains without the logistical challenges of other liquid cooled solutions.

At the University, the Iceotope Solution currently operates contaminant free in a large thermofluids laboratory, using a total of 100W of pump power to harvest the heat from the IT systems without the requirement for fans.

The fact that the system does not require any fans (none in the servers or power supply units nor any in the heat rejection system within the laboratory) was actually a deployment requirement for the University because its current installed location does not offer any control over harmful IT contaminants.

The Iceotope Modules are sealed units, encapsulating the servers; therefore dust, humidity and air pollution have no effect. Doing away with fans also means the system runs silent, so it can be placed in densely populated locations. At the University of Leeds, the Iceotope Solution is located in the middle of a busy research facility and has several academic teams working in close proximity at all times, without it being a noisy distraction.

Iceotope Module with 3M™ Novec™ Engineered Fluid

Unique to Iceotope technology, the system also allows for reuse of waste heat. In the case of the University’s deployment, the hot water leaving the Iceotope cabinet passes through a set of large domestic hot water radiators to help warm a large open plan laboratory. All the while, the servers are able to run room neutral in terms of heat because they are constantly immersed in 3MTM NovecTM fluid, which conducts the heat away from the servers to the cabinet.

The reuse of heat in this deployment is enabled using two domestic hot water radiators and a central heating pump. The pump, together with the two pumps in the cabinet’s primary hot water circuit, require no more than 100W of power to transfer 2.5kW of heat and can take input water of around 50°C to 55°C.

Case Study ResultsIt was determined that further deployment of the Iceotope Solution at the University of Leeds would offer a considerable financial, resource and efficiency savings, particularly if the High

Iceotope Solution at University of Leeds

Radiators at University of Leeds

Performance Computing Facility Operators decided to follow the School of Mechanical Engineering’s lead and install similar systems. Furthermore, the installation of the Iceotope system now allows the University to continue its research into the scientific and engineering grand challenges via HPC without being a contributing factor itself.

Installation Requirements:

Augment current High Performance Computing (HPC) facility offering at Leeds; in particular be able to run large thermal Computational Fluid Dynamics (CFD) simulations.

Demonstrate strong energy savings and become an experimental apparatus situated in the theromofluids laboratory for research into low grade heat re-use.

Install in a large open plan laboratory where there is no control over material that may be classed as harmful.

Run off a maximum of two 16 amp 3 phase, with the potential to install UPS (up to 25kVA).

Energy Savings:

The Leeds Iceotope cabinet currently has 11 systems with a monitored power use of 2.3kW with 95W (80W standard configuration plus a 15W pump) to reject the heat into the laboratory through domestic radiators.

If installed in the Leeds HPC data centre 2.3kW would require 885W of cooling (based on 260/100kW) – an 89% saving on cooling energy consumption.

Extrapolating to 48 systems – 10kW would require around 120W of pump power to reject heat into the laboratory versus 3.85kW of cooling power if installed in the HPC data centre – providing a potential 97% saving on cooling energy consumption.

Thermal Performance – Iceotope Solution:

Iceotope Solution at University of Leeds

Thermostat Graphic

System temperatures are at about 50° to 55°C. CPUs are up to 70° to 80°C.

Two socket systems designed for air and showing 3° to 6°C difference.

Thermal Performance – Heat Rejection(Used to heat lab):

Supply and return temperature to cabinet heat exchanger is 6°C. Flow rate in the radiator circuit is

120 litres/hour.

Rate of heat transfer to the laboratory is 830W for a situation of 1889W consumed. Heat transfer process loses 56% via other surfaces.

Similar losses for 48 units at 10kW = 11% losses.

Insulation (unusual for Data Centers)

Performance of Systems in Liquid Environment:

High Performance Linpack compiled against AVX (and FMA4 for AMD) was run to determine the actual Gflops performance.

For Intel E5-2620 CPUs – performance of 192GFlops was obtained using 180W of power for the performance test.

For the AMD Opteron 6272 – performance of 138GFlops was obtained using 255W of power when the test was running.

Scaling out the system to 48 units on the E5-2620 CPU gives a potential facility performance of 1.07GFlops/Watt (possible 36th position in the Green500).

Conclusions:

The first Iceotope Solution using 3M™ Novec™ Engineered Fluid is installed in a large laboratory in

Thermal Photos Iceotope Solution

Radiator Gauges

Computational Fluid Dynamics Simulations of Iceotope Module

the University of Leeds.

System is a small production HPC system that is also an experimental apparatus running services

Energy savings on cooling are between 89 and 97% over a traditional HPC cooled system.

Heat is re-used to provide supplementary heating to a large open laboratory space.

About the University of Leeds

The 2008 Research Assessment Exercise showed the University of Leeds to be the UK's eighth biggest research powerhouse. It is one of the largest higher education institutions in the UK and a member of the Russell Group of research-intensive universities. The University's vision is to secure a place among the world's top 50 by 2015. www.leeds.ac.uk

About Iceotope

Iceotope R&D Ltd has been established to develop and deliver “full time free cooling for ICT anywhere”. This is done by combining next generation liquid cooling technology, industry standard ICT and Iceotope’s own I.P. Designed with environmental impact in mind, this is the first truly sustainable and resource efficient solution to ICT cooling. Iceotope’s patented energy efficient technology is designed, engineered and manufactured in the UK.www.iceotope.com

About 3M

3M captures the spark of new ideas and transforms them into thousands of ingenious products. Our culture of creative collaboration inspires a never-ending stream of powerful technologies that make life better. 3M is the innovation company that never stops inventing. With $30 billion in sales, 3M employs about 88,000 people worldwide and has operations in more than 70 countries.www.3M.com

Prepared by:Iceotope R&D LtdAMP Technology CentreBrunel WayRotherham, S60 5WGUnited Kingdom

Telephone: 0844 225 7101www.iceotope.com

Registered in United KingdomCompany Number 08004482© Iceotope 2013E&OE All Rights Reserved

Iceotope Solution