Communication and cooling aware job allocation in data centers for communication-intensive workloads

Abstract

Energy consumption is an increasingly important concern in data centers. Today, nearly half of the energy in data centers is consumed by the cooling infrastructure. Existing policies on thermally-aware workload allocation do not consider applications that include many tasks (or threads) running on a large set of nodes with significant communication among the tasks. Such jobs, however, constitute most of the cycles in high performance computing (HPC) domain, and have started to appear in other data centers as well. Job allocation strongly affects the performance of such communication-intensive applications. Communication-aware job allocation methods exist, but they focus solely on performance and do not consider cooling energy. This paper proposes a novel job allocation methodology to jointly minimize communication cost and cooling energy consumption in data centers. We formulate and solve the joint optimization problem using binary quadratic programming. Our joint optimization algorithm reduces cooling energy by 16.4% on average with only a 2.66% average increase in application running time compared to solely performance-aware allocations. To further optimize the communication cost, we develop a Charm++ based framework that extracts the communication behavior of applications. We then integrate our job allocation policy with recursive coordinate bisection (RCB) based task mapping method to place highly-communicating tasks in close proximity. Experimental results show that task mapping further decreases the communication cost by up to 20.9% compared to assuming all-to-all communication, a popular assumption in much of the prior work. 20 temperature is sufficient to cool the data center [4,5].Performance is another first-order constraint in data center design and management. Communication-intensive parallelized applications constitute most of HPC workloads, and have started to occupy more cycles in other data centers following the efforts in designing parallelized large-scale applications in many domains. These applications run 25 on multiple nodes with intensive communication between the threads, with job running times in the range of minutes, hours, or even days. In particular, the performance of a communication-intensive application is sensitive to the specific allocation and mapping onto the compute nodes. In other words, communication cost has a significant impact on system performance [6,7,8], with long communication distances and communication 30 resource contention increasing the overall delay. Leung et al. have shown a 2X slow down for two communication-intensive jobs when hand-placing them such that their communication paths overlap significantly [6].Most of the existing performance-aware job allocation algorithms focus on minimizing the average number of communication hops between communicating nodes (e.g., [7,8,35 9, 10]). For data centers running transactional enterprise loads, performance-aware job allocation algorithms focus on satisfying the response time constraints imposed by service level agreemen...

Performance optimality or reproducibility

“…There is little research that explores the simultaneous impact of network, power, and concurrency in real systems [50]. This limited research is either focused on power and IO management [14,64] or on temperatureawareness for communication-intensive workloads [46,47]. Our work is the first-of-its-kind study to support a truly multi-objective environment and to quantitatively evaluate various configurations on a real system while understanding the influence scores of different parameters.…”

Spatio-temporal thermal-aware scheduling for homogeneous high-performance computing datacenters

“…Chavan et al [28] proposed TIGER, a thermalaware technique specifically designed to reduce the cooling cost due to storage systems in datacenters. Meng et al [29] considered communication-bound HPC applications and studied joint optimization of cooling and communication costs via job allocation. Piaţek [30] studied thermal-aware load balancing with fan management in air-cooled server systems in order to improve the energy efficiency.…”

High Performance Communication on Reconfigurable Clusters

“…We have investigated which applications are particularly suitable for mapping onto FPGA clusters (Meng et al, 2016), and find that the 3D FFT is a good candidate. We fully implement the 3D FFT and use it as a case study to show how an application is mapped onto our cluster and communication infrastructure.…”