Beyond the CPU: Hardware Performance Counter Monitoring on Blue Gene/Q

Abstract

The Blue Gene/Q (BG/Q) system is the third generation in the IBM Blue Gene line of massively parallel, energy efficient supercomputers that increases not only in size but also in complexity compared to its Blue Gene predecessors. Consequently, gaining insight into the intricate ways in which software and hardware are interacting requires richer and more capable performance analysis methods in order to be able to improve efficiency and scalability of applications that utilize this advanced system. The BG/Q predecessor, Blue Gene/P, suffered from incompletely implemented hardware performance monitoring tools. To address these limitations, an industry/academic collaboration was established early in BG/Q's development cycle to insure the delivery of effective performance tools at the machine's introduction. An extensive effort has been made to extend the Performance API (PAPI) to support hardware performance monitoring for the BG/Q platform. This paper provides detailed information about five recently added PAPI components that allow hardware performance counter monitoring of the 5D-Torus network, the I/O system and the Compute Node Kernel in addition to the processing cores on BG/Q. Furthermore, we explore the impact of node mappings on the performance of a parallel 3D-FFT kernel and use the new PAPI network component to collect hardware performance counter data on the 5D-Torus network. As a result, the network counters detected a large amount of redundant inter-node communications, which we were able to completely eliminate with the use of a customized node mapping.

Investigating power capping toward energy‐efficient scientific applications

“…The PAPI performance monitoring library provides a coherent methodology and standardization layer to performance counter information for a variety of hardware and software components, including CPUs, graphics processing units (GPUs), memory, networks,() I/O systems, power systems,() and virtual cloud environments . PAPI can be used independently as a performance monitoring library and tool for application analysis; however, PAPI finds its greatest utility as a middleware component for a number of third‐party profiling, tracing, and sampling toolkits (eg, CrayPat, HPCToolkit, Scalasca, Score‐P, TAU, Vampir, PerfExpert), making it the de facto standard for performance counter analysis.…”

Verified Instruction-Level Energy Consumption Measurement for NVIDIA GPUs

“…Line [10] sets the loop count to 1M iterations. The loop body, lines [13][14][15][16][17][18][19][20][21][22][23][24][25][26][27], is composed of 5 back-to-back unsigned div instructions with dependencies, to make sure that the compiler does not optimize any of them. We do a load-add-store operation on the output of the 5 th div operation and begin the loop with new values each time to force the compiler to execute the instructions.…”

Experiences in autotuning matrix multiplication for energy minimization on GPUs

“…The increasing focus on resource efficiency has motivated hardware manufacturers to start embedding energy counters in the processors themselves [12,13]. Profiling tools like performance application programming interface [14] provide user interfaces to these metrics, and allow developers to correlate the power and energy consumption to other application-specific metrics [15]. Similarly, when Cray introduced the XC-30 line, they included the Power Management Data Base 5098 H. ANZT ET AL.…”