Combining One-Sided Communications with Task-Based Programming Models

Abstract

Hybrid programming combining task-based and message-passing models is an increasingly popular technique to exploit multi-core clusters. The Task-Aware MPI (TAMPI) library integrates both models enabling the safe overlap of computation and communication tasks using two-sided MPI communications. Two-sided primitives combine data transfers with implicit synchronizations, but one-sided models usually offer more efficient data transfers decoupling synchronizations. MPI offers four distinct one-sided synchronization modes, while GASPI is a PGAS API providing one-sided operations with remote notifications for fine inter-process synchronizations.In this paper, we study the challenges of integrating MPI and GASPI one-sided operations with the OpenMP and OmpSs-2 tasking models. We propose and implement several extensions to the GASPI and OmpSs-2 programming models, which are leveraged by a new library called Task-Aware GASPI (TAGASPI). The TAGASPI library allows the efficient and safe use of one-sided operations with remote notifications inside tasks. Both TAGASPI and TAMPI transparently manage communications issued by tasks and allow these to overlap with computation tasks naturally, following a data-flow model. These libraries are complementary and can be mixed in the same application.Our experience porting several mini-apps to this hybrid model shows that TAGASPI helps leverage one-sided communications with similar complexity to pure and hybrid two-sided MPI approaches. We show that our hybrid one-sided approach outperforms the pure MPI strategies, but it also surpasses the TAMPI's performance when stressing communication phases, e.g., increasing the communication parallelism and reducing the communication tasks' sizes.

OmpSs-2@Cluster: Distributed Memory Execution of Nested OpenMP-style Tasks

“…While there are a variety of approaches within the node, described by the "X", such as OpenMP, OmpSs, OpenACC and others, the de facto standard for programming multiple nodes is MPI. In all cases the program must combine two fundamentally different programming models, which is difficult to get right [30,29]. The tasking approach of OpenMP and OmpSs offers an open, clean and stable way to improve hardware utilization through asynchronous execution while targeting a wide range of hardware, from SMPs, to GPUs, to FPGAs.…”