Multi-accelerator extension in OpenMP based on PGAS model

Abstract

Many systems used in HPC field have multiple accelerators on a single compute node. However, programming for multiple accelerators is more difficult than that for a single accelerator. Therefore, in this paper, we propose an OpenMP extension that allows easy programming for multiple accelerators. We extend existing OpenMP syntax to create Partitioned Global Address Space (PGAS) on separated memories of several accelerators. The feature enables users to perform programming to use multiple accelerators in ease. In performance evaluation, we implement the STREAM Triad and the HIMENO benchmarks using the proposed OpenMP extension. As a result of evaluating the performance on a compute node equipped with up to four GPUs, we confirm that the proposed OpenMP extension demonstrates sufficient performance.

A Novel Set of Directives for Multi-device Programming with OpenMP

“…There has been a variety of studies on the problem of work distribution among multiple accelerators with OpenMP, e.g. [22], [23]. There, for example, the authors tried to provide a handful of distribution strategies that could eventually fit to real-world programs where the distribution of data is more complex than simply splitting the data among the available devices.…”