A 'cool' way of improving the reliability of HPC machines

Abstract

Soaring energy consumption, accompanied by declining reliability, together loom as the biggest hurdles for the next generation of supercomputers. Recent reports have expressed concern that reliability at exascale level could degrade to the point where failures become a norm rather than an exception. HPC researchers are focusing on improving existing fault tolerance protocols to address these concerns. Research on improving hardware reliability, i.e., machine component reliability, has also been making progress independently. In this paper, we try to bridge this gap and explore the potential of combining both software and hardware aspects towards improving reliability of HPC machines. Fault rates are known to double for every 10• C rise in core temperature. We leverage this notion to experimentally demonstrate the potential of restraining core temperatures and load balancing to achieve two-fold benefits: improving reliability of parallel machines and reducing total execution time required by applications. Our experimental results show that we can improve the reliability of a machine by a factor of 2.3 and reduce the execution time by 12%. In addition, our scheme can also reduce machine energy consumption by as much as 25%. For a 350K socket machine, regular checkpoint/restart fails to make progress (less than 1% efficiency), whereas our validated model predicts an efficiency of 20% by improving the machine reliability by a factor of up to 2.29.

Maximizing Throughput of Overprovisioned HPC Data Centers Under a Strict Power Budget

“…We also plan to provide rich support for user priorities in PARM. Thermal behavior of CPUs can significantly affect the reliability of a machine [44] as well as the cooling costs of the data center [45]. We also plan to investigate the possibility of incorporating thermal constraints along with a strict power constraint in our scheduling scheme.…”

Spatio-temporal thermal-aware scheduling for homogeneous high-performance computing datacenters

“…(2) To reduce the failure rate of the processors for reliability concerns [2]. (3) To avoid hardware interrupt triggered by some chips when the temperature exceeds a certain redline value, which can cause severe performance degradation and energy increase [25].…”

“…The objective is to minimize the makespan for a set of computation-intensive applications subject to a temperature threshold, which cannot be violated at any time during the execution. Indeed, such a threshold is imposed in many resource management systems for either energy reduction considerations or reliability concerns [2,3]. To tackle this problem, we introduce a novel notion, called thermal-aware load, to capture more precisely the loads of the servers under the thermal constraint.…”

Parallel Programming with Migratable Objects: Charm++ in Practice

“…To reduce this, the computer room air conditioning (CRAC) temperature can be set to a higher degree. However, higher room temperature can cause overheating of cores which reduces hardware reliability [9]. Dynamic Voltage Frequency Scaling (DVFS) is commonly used to prevent overheating by modulating chip frequency and voltage.…”