Exploring Programming Multi-GPUs Using OpenMP and OpenACC-Based Hybrid Model

Abstract

Heterogeneous computing come with tremendous potential and is a leading candidate for scientific applications that are becoming more and more complex. Accelerators such as GPUs whose computing momentum is growing faster than ever offer application performance when compute intensive portions of an application are offloaded to them. It is quite evident that future computing architectures are moving towards hybrid systems consisting of multi-GPUs and multicore CPUs. A variety of high-level languages and software tools can simplify programming these systems. Directive-based programming models are being embraced since they not only ease programming complex systems but also abstract low-level details from the programmer. We already know that OpenMP has been making programming CPUs easy and portable. Similarly, a directive-based programming model for accelerators is OpenACC that is gaining popularity since the directives play an important role in developing portable software for GPUs. A combination of OpenMP and OpenACC, a hybrid model, is a plausible solution to port scientific applications to heterogeneous architectures especially when there is more than one GPU on a single node to port an application to. However OpenACC meant for accelerators is yet to provide support for multi-GPUs. But using OpenMP we could conveniently exploit features such as for and section to distribute compute intensive kernels to more than one GPU. We demonstrate the effectiveness of this hybrid approach with some case studies in this paper.

Endpoint Security in Networks: An OpenMP Approach for Increasing Malware Detection Speed

“…In order to add parallel capacity for systems based on multi-core CPU, there are different platforms and libraries available-for example, Intel (Folsom, CA, USA) Thread Building Blocks (TBB) [9,10], Pthreads [10], Message Passing Interface (MPI) [11], OpenMP [11,12], Open Computing Language (OpenCL) [13] and Open Accelerators (OpenACC) [14]. All of them can be used for programming multi-core CPUs, but MPI is more appropriate for distributed shared memory (DSM) systems, such as computational clusters [11].…”

“…Although presenting the advantage of a unified view of the system memory space, OpenMP does not have a better performance than MPI when used on heterogeneous architectures with non-uniform memory access (NUMA) [11,12]. Since version 4.0, OpenMP has supported programming for GPU [14].…”

Supporting multiple accelerators in high-level programming models

“…Although these high-level programming models lack such [20] used OpenMP + OpenACC to utilize multiple GPUs attached in a multi-core system (a single node of a cluster) and explained how to do the inter-GPU communication with OpenMP synchronization and OpenACC data synchronization primitives. Hart et al [8] used Co-Array Fortran (CAF) + OpenACC and Levesque et al [10] used MPI + OpenACC to utilize multi-GPU in GPU cluster.…”

Developing adaptive multi-device applications with the Heterogeneous Programming Library

“…The exploitation of heterogeneous parallelism across the devices existing in a node by combining OpenMP and OpenACC, or ad-hoc directives has been explored in [26] and in [5], respectively. The result and the differences with respect to HPL are similar to those of OmpSs, with the addition that these solutions do not automatically schedule and synchronize the tasks based on their data dependencies.…”