Performance Assessment of OpenMP Compilers Targeting NVIDIA V100 GPUs

Abstract

Heterogeneous systems are becoming increasingly prevalent. In order to exploit the rich compute resources of such systems, robust programming models are needed for application developers to seamlessly migrate legacy code from today's systems to tomorrow's. Over the past decade and more, directives have been established as one of the promising paths to tackle programmatic challenges on emerging systems. This work focuses on applying and demonstrating OpenMP offloading directives on five proxy applications. We observe that the performance varies widely from one compiler to the other; a crucial aspect of our work is reporting best practices to application developers who use OpenMP offloading compilers. While some issues can be worked around by the developer, there are other issues that must be reported to the compiler vendors. By restructuring OpenMP offloading directives, we gain an 18x speedup for the su3 proxy application on NERSC's Cori system when using the Clang compiler, and a 15.7x speedup by switching max reductions to add reductions in the laplace mini-app when using the Cray-llvm compiler on Cori.

Single‐ and multi‐GPU computing on NVIDIA‐ and AMD‐based server platforms for solidification modeling application

“…A significant challenge is to port existing applications to platforms with accelerators, including GPUs. The ultimate goal 39 is taking advantage of these powerful platforms without having to learn the hardware details or significantly change the application codes. Numerous programming models and environments have been developed, including CUDA, 11 HIP, 12 OpenCL, 24 and Kokkos 40 .…”

“…An alternative option is directive‐based models such as OpenACC 41 and OpenMP 42 . They offer 39 an abstraction layer over different hardware types with a unified interface that allows reducing the work needed to accelerate applications, requiring only some "hints" or annotations to be added for the compiler. The OpenMP model incorporated support for offloading code to accelerators from version 4.0 (released in 2013) and has upgraded various features in subsequent versions (till version 5.2).…”

“…The major advantage of this approach is applying the same programming interface for the whole application. Currently, offloading code to NVIDIA GPUs is supported by using GCC, Clang, IBM XL, HPE, and NVIDIA HPC compilers 39,51 . In this work, the Clang compiler is used.…”

Evaluating GPU Programming Models for the LUMI Supercomputer

“…For many programming models, there are studies that evaluate these for CPUs or GPUs. In [7] the authors study OpenMP offload on NVIDIA V100 with a few mini-apps and various compilers, observe performance variations, and provide some OpenMP optimization techniques. In [8], the authors present the computebound mini-app miniBUDE and evaluate various programming models, including offload to GPUs.…”

OpenACC Acceleration of an Agent-Based Biological Simulation Framework

“…The offloading model is beginning to mature as we speak based on the validation and verification findings [27], [28]. We also note that the model is being used on mini-applications [29] and other applications including Pseudo-Spectral Direct Numerical Simulation-Combined Compact Difference (PSDNS-CCD3D) [30] and Quicksilver [31] among others [32], [33], [34], [30], [35]. We chose to go with OpenACC instead of the OpenMP offloading model.…”