Flow network analysis of the IBM Power 775 supercomputer Water Cooling System

Abstract

In 2011 IBM announced the Power 775 Supercomputing node/system which, for the time, was a monumental leap forward in computing performance and energy efficiency. The system was designed from the start with water cooling in mind. The result: a system with greater than 95% of its heat load conducted directly to water and a system that, together with a rear door heat exchanger, removes 100% of its heat load to water with no requirement for room air conditioning. In addition to direct water cooling the processor, the memory, power conversion, and I/O electronics also conduct their heat directly to water. Also included within the framework of the system is a disk storage unit (i.e. disc enclosure) containing an interboard air-to-water heat exchanger.A detailed flow network analysis was undertaken to assure adequate flow to all components in the system while minimizing pump power consumption for a wide range of system configurations. The analysis was validated by both early development experimentation as well as final system verification. The network analysis could therefore be used to establish pump speed tables for normal and abnormal modes of operation. This paper will outline the assumptions and methodology associated with the flow network analysis and demonstrates how the analysis was used for the design and operation of the water cooling system. NOMENCLATUREA cross sectional area, m 2 BPA bulk power conversion assembly C1 flow impedance coefficient, N-s/m 5 C2 flow impedance coefficient, N-s 2 /m 8 CEC central electronics complex DCCA distributed converter and control assembly ¡P differential pressure, Pa EPDM ethylene propylene diene monomer (M-class) rubber FIFO first in, first out FNM Flow Network Modeling gpm gallons per minute (1 gpm = 6.30902E-5 m 3 /s) ID inner diameter in inches k loss coefficient kPa kilopascals (10 3 Pa) OD outer diameter Pa Pascal (unit of pressure = 1 N/m 2 ) Q volumetric flow rate, m 3 /s RDHx rear door heat exchanger RPM rotations per minute (pump speed) WCU water conditioning unit Greek symbols ρ mass density (kg/m 3 )

Rack Level Study of Hybrid Liquid/Air Cooled Servers: The Impact of Flow Distribution and Pumping Configuration on Central Processing Units Temperature

“…Providing sufficient flow rate and distributing the flow equally for all the servers in the rack is necessary to maintain the CPU temperatures within their designed conditions. Ellsworth [19] analysed the flow distribution using MacroFlow software in the IBM 775 rack of supercomputer servers. The rack contained 12 servers, 2 bulk power assemblies, a rear door heat exchanger and four pumps connected in parallel.…”