XcalableACC: Extension of XcalableMP PGAS Language Using OpenACC for Accelerator Clusters

Abstract

The present paper introduces the XcalableACC (XACC) programming model, which is a hybrid model of the XcalableMP (XMP) Partitioned Global Address Space (PGAS) language and OpenACC. XACC defines directives that enable programmers to mix XMP and OpenACC directives in order to develop applications that can use accelerator clusters with ease. Moreover, in order to improve the performance of stencil applications, the Omni XACC compiler provides functions that can transfer a halo region on accelerator memory via Tightly Coupled Accelerators (TCA), which is a proprietary network for transferring data directly among accelerators. In the present paper, we evaluate the productivity and the performance of XACC through implementations of the HIMENO Benchmark. The results show that thanks to the productivity improvements, XACC requires less than half the source lines of code compare to a combination of Message Passing Interface (MPI) and OpenACC, which is commonly used together as a typical programming model. As a result of these performance improvements, XACC using TCA achieved up to 2.7 times faster performance than could be obtained via the combination of OpenACC and MPI programming model using GPUDirect RDMA over InfiniBand.

MACC: An OpenACC Transpiler for Automatic Multi-GPU Use

“…Also, programming models targeting accelerator clusters are proposed [17,18]. These models provide explicit functions to distribute computations over multiple accelerators.…”

Heterogeneous distributed computing based on high‐level abstractions

“…As a result, it shares similar limitations; the most important difference being that it allows to define distributed arrays as a collection of tiles located in different nodes but lacking all the tile‐level semantics, advanced syntax, collective manipulation capabilities, and data‐parallel operations of H 2 TAs. Another interesting alternative that relies on language extensions and compiler directives is XcalableACC . The XMP extensions are in charge of providing distributed arrays with a small subset of the array operations of H 2 TAs and without their tile‐level features.…”

“…Another interesting alternative that relies on language extensions 30 and compiler directives is XcalableACC. 31 The XMP extensions are in charge of providing distributed arrays with a small subset of the array operations of H 2 TAs and without their tile-level features. As for heterogeneity, the fact that XcalableACC relies on OpenACC 32 reduces its portability compared to OpenCL, 33 which we use as backend, and sometimes also the performance, as OpenACC has been found to often offer considerably less performance than manually optimized kernels.…”

Overlapping Data Transfers with Computation on GPU with Tiles

“…Our approach utilizes CUDA in data transfers for better performance but leverages an OpenACC compiler to generate efficient kernel code. XACC [22], a directive based language that targets multi-GPUs also leverages OpenACC for kernel code generation however XACC does not provide asynchronous data transfers to overlap data movement with computation.…”