A Novel Multi-level Integrated Roofline Model Approach for Performance Characterization

Abstract

With energy-efficient architectures, including accelerators and many-core processors, gaining traction, application developers face the challenge of optimizing their applications for multiple hardware features including many-core parallelism, wide processing vector-units and onchip high-bandwidth memory. In this paper, we discuss the development and utilization of a new application performance tool based on an extension of the classical roofline-model for simultaneously profiling multiple levels in the cache-memory hierarchy. This tool presents a powerful visual aid for the developer and can be used to frame the many-dimensional optimization problem in a tractable way. We show case studies of real scientific applications that have gained insights from the Integrated Roofline Model.

A CAD-based methodology to optimize HLS code via the roofline model

“…Over the years, the Classical Roofline model [36] has been formulated for multicore [19,23] and GPU [18,40] architectures. Moreover, assisted methodologies and automatic tools [5,22,28,29] have been introduced to ease Roofline model generation for scientific and HPC application optimization.…”

“…Condensing the optimization space in a single performance figure, this model provides intuitive guidance to optimize complex applications. In this way, the Roofline model has become a confirmed methodology to optimize HPC applications targeting multicore [19,23] and GPU [18,40] architectures. With Field-Programmable Gate Array (FPGA) devices becoming an appealing solutions to accelerate HPC applications, a dual Roofline model for reconfigurable devices is becoming of real interest.…”

Hierarchical Roofline analysis for GPUs: Accelerating performance optimization for the NERSC‐9 Perlmutter system

“…Previously, the Roofline model was expanded to support the full memory hierarchy by adding additional bandwidth “ceilings.” Similarly, additional ceilings beneath the Roofline can be added to represent performance bottlenecks arising from lack of vectorization or the failure to exploit fused multiply‐add (FMA) instructions.…”

“…Orthogonal to the Roofline description of hardware is characterizing applications in terms of Roofline‐related coordinates, ie, Performance (GFLOP/s) and Arithmetic Intensity (FLOPs/Byte). One can employ a variety of methods to calculate these terms ranging from hand counting FLOPs and estimating bytes, to performance counters, to software simulators that trade performance for accuracy.…”

“…Over the last decade, Roofline analysis has been proven to be a great success, especially with the hierarchical Roofline on Intel CPUs and was a benefit to understanding performance on NERSC's previous KNL‐based Cori Supercomputer . However, Roofline has yet to be fully developed on NVIDIA GPUs.…”