Towards fine-grained dynamic tuning of HPC applications on modern multi-core architectures

Abstract

There is a consensus that exascale systems should operate within a power envelope of 20MW. Consequently, energy conservation is still considered as the most crucial constraint if such systems are to be realized. So far, most research on this topic focused on strategies such as power capping and dynamic power management. Although these approaches can reduce power consumption, we believe that they might not be sufficient to reach the exascale energy-efficiency goals. Hence, we aim to adopt techniques from embedded systems, where energy-efficiency has always been the fundamental objective. A successful energy-saving technique used in embedded systems is to integrate fine-grained autotuning with dynamic voltage and frequency scaling. In this paper, we apply a similar technique to a real-world HPC application. Our experimental results on a HPC cluster indicate that such an approach saves up to 20% of energy compared to the baseline configuration, with negligible performance loss.

Energy-Aware High-Performance Computing: Survey of State-of-the-Art Tools, Techniques, and Environments

“…(2) DVFS/DFS/DCT [49] For MPI applications with the goal not to impact performance [47] Uniform frequency power-limiting investigates results for the fixed frequency mode, minimum power level assigned to a job, and automatic mode with consideration of available power [45] Core and uncore frequency scaling of CPUs [55,56] Minimization of energy usage through DVFS on particular nodes [52] DFS, DCT [14] DVFS, DCT [37] Control of frequency on a GPU [41] DVFS with dynamic detection of computation phases (memory and CPU bound) [59] DVFS with a posteriori (using logs) detection and prioritization of computation phases (memory and CPU bound) [61] Sysfs interface is used [63,64] DCT, combined DVFS/DCT [65] Sysfs interface [72] Setting the frequency according to the established computing center policies…”

“…In terms of system components that can be controlled in terms of power and energy, the literature distinguishes frequency, core and uncore [45], disk [53], and network [53]. e latter can also be done through Energy-Efficient Ethernet (EEE) [78] that can turn physical layer devices into a low-power mode with savings up to even 70%-work [78] shows that the overhead of the technology is negligible for many practical scenarios.…”

“…(2) Multiprocessor system [44] Task scheduling with thermal consideration for a heterogeneous real-time multiprocessor system-onchip (MPSoC) system [30] Presents a hybrid approach PUPiL (Performance under Power Limits)-a hybrid software/hardware power capping system based on a decision framework going through nodes and making decisions on configuration, considered for single and multiapplication scenarios (cooperative and oblivious applications) [45] With notes specific to clusters [14] Systems with 2 socket Westmere-EP, 2 socket Sandy Bridge-EP, and 1 socket Ivy Bridge-HE CPUs [46] Dual-socket server with two Intel Xeon CPUs Table 2: Continued.…”

“…(2) Multiprocessor system [44] A heterogeneous real-time multiprocessor system-on-chip (MPSoC) system-consists of a number of processors each of which runs at its voltage and speed [47] A cluster with Intel Xeon CPUs [30] A multiprocessor system with Intel Xeon CPUs [48] A cluster with ARM CPUs, a cluster with Intel Ivy Bridge CPUs [74] A grid system parametrized with the number of hosts, distribution of computing capacities, and host selection policy [50] A system with a number of nodes with multicore CPUs assumed in the simulated HPC platform and cores of an Intel core M CPU with 6 voltage/frequency levels assumed [53] Cluster with consideration of CPU, memory, disk, and network [55,56,61,62,65] Cluster with CPUs [45] Many cores within a system, and core and uncore frequencies are of interest [57] With consideration of disk and network scaling [14] Systems with 2 socket Westmere-EP, 2 socket Sandy Bridge-EP, and 1 socket Ivy Bridge-HE CPUs [76] Undefined machines in a data center capable of hosting up to 15 VMs [60] Homogeneous multicore cluster [63,64] Cluster with multicore CPUs [66,67] Cluster in a data center [68] Sandy Bridge cluster [69] Cluster with InfiniBand [46] Dual-socket server with two Intel Xeon CPUs [70] Overprovisioned HPC cluster with CPUs [71] Cluster with 1056 Dell PowerEdge SC1425 nodes…”

“…Applications/benchmarks [33] Mibench and mediabench [44] Automotive-industrial, consumer-networking, telecom, mpeg [73] MD5 password breaking application [47] Monte Carlo simulation of particle transport unstructured implicit finite element method molecular dynamics unstructured shock hydrodynamics [30] Both single-application and multiapplication workloads PARSEC (x264, swaptions, vips, fluidanimate, Black Scholes, bodytrack), Minebench (ScalParC, kmeans, HOP, PLSA, svmfe, btree, kmeans_fuzzy), Rodinia (cfd, nn, lud, particlefilter), Jacobi, swish++, dijkstra [48] CoMD, Lulesh, MP2C [34] NAS parallel benchmarks (NPB) kernel EP, European option pricing benchmark Black Scholes [74] Workload simulated in a scheduling simulator [49] Matrix multiplication + P_write_priv benchmark-an I/O benchmark of IMB package [50] Job models from historical data from HLRS (highperformance computing center Stuttgart) [51] Early evaluation through an application extracted from Drug Discovery code, computation of interatomic distances and overlap of drug molecule, and protein active site probes [52] HPCG, NAS parallel benchmarks, NICAM-DC-MINI (for Post-K Japanese national flagship supercomputer development) [53] For system adaptation experiments: Molecular dynamics simulation (MDS), advance research Weather research and forecasting (WRF-ARW) [35] miniMD (parallel molecular dynamics code), Jacobi-3D stencil computation code [54] Parallel aerospace application PMLB (Lattice Boltzmann), parallel earthquake simulation eq3dyna [36] GEM software-calculation of electrostatic potentials generated by charges within molecules [75] Generation of parity data using GPUs [56] MILC, SWEEP3D, CG, FT, MG, LAMMPS, POP [55] MILC, GTC, SWEEP3D, PSCYEE, LBMHD, NAS parallel benchmarks-intratask scaling, LAMMPS, HYCOM, WRF, POP-intertask scaling [37] Selected MAGMA and Rodinia benchmarks [38] Synthetic sporadic real-time tasks used for evaluation of performance [45] Mega lattice Site updates per Second [39] Virusdetectioncl, NVIDIA CUDA SDK, GP GPU-Sim, Rodinia [57] IOzone, iperf, stream, stress (single node), a set of NPB benchmark (cluster): CG, MG, MDS, WRF-ARW, POP X1 benchmark, GeneHunter [14] NAS OpenMP parallel benchmarks…”