Achieving High Performance on Supercomputers with a Sequential Task-based Programming Model

Abstract

The emergence of accelerators as standard computing resources on supercomputers and the subsequent architectural complexity increase revived the need for high-level parallel programming paradigms. Sequential task-based programming model has been shown to efficiently meet this challenge on a single multicore node possibly enhanced with accelerators, which motivated its support in the OpenMP 4.0 standard. In this paper, we show that this paradigm can also be employed to achieve high performance on modern supercomputers composed of multiple such nodes, with extremely limited changes in the user code. To prove this claim, we have extended the StarPU runtime system with an advanced inter-node data management layer that supports this model by posting communications automatically. We illustrate our discussion with the task-based tile Cholesky algorithm that we implemented on top of this new runtime system layer. We show that it enables very high productivity while achieving a performance competitive with both the pure Message Passing Interface (MPI)-based ScaLAPACK Cholesky reference implementation and the DPLASMA Cholesky code, which implements another (non-sequential) task-based programming paradigm.

Exploiting Data Sparsity for Large-Scale Matrix Computations

“…Fig. 9 presents the execution traces of dense and TLR Cholesky factorizations, as implemented in task-based Chameleon [1] and HiCMA, respectively. These traces highlight the CPU idle time (red color) in HiCMA, since StarPU is not able to compensate the data movement overhead with the tasks' computations.…”

“…HiCMA leverages the tile data descriptor in order to support the new tile low-rank (TLR) compression format. While this data descriptor is paramount to expose parallelism, it is also critical for the data management in distributed-memory environment [1,12]. HiCMA adopts a flattened algorithmic design to bring to the fore the task parallelism, as opposed to plain recursive approach, which has constituted the basis for performance of previous H-matrix libraries [19,17,16].…”

Asynchronous Task-Based Polar Decomposition on Single Node Manycore Architectures

“…This is achieved by standardizing existing dynamic runtime system APIs (e.g., OpenMP [32], OmpSs [33], [34], [35], QUARK [36], StarPU [37], [43], PaRSEC [38], SuperMatrix [39]) through a thin layer of abstraction, making the user developer experience oblivious to the underneath runtime system and its corresponding hardware deployment. For instance, this hardware/runtime-oblivious software infrastructure has been already used with StarPU [40], and more recently with OmpSs [41], in the context of computational astronomy applications.…”

“…This improves the user productivity, and it is even more realistic for runtimes such as StarPU, which are able to transparently handle single heterogeneous nodes, and eventually multiple heterogeneous nodes in case the StarPU-MPI [43] extension is used. To enable such portability, StarPU tasks are associated to codelets which groups under the same name multiple implementations of the same task: CPU, CUDA, OpenCL, OpenMP, etc.…”

Evaluation of dataflow programming models for electronic structure theory

“…One of StarPU's strengths is that the system relies on the variety of scheduling strategies available to adapt to applications and platforms with both centralized and distributed solutions. Recently, support for automatically inferring data communication was added to StarPU to help users moving toward distributed architectures …”