Productive Programming of GPU Clusters with OmpSs

Bueno, Javier; Planas, Judit; Durán, Alejandro; Badía, Rosa M.; Martorell, Xavier; Ayguadé, Eduard; Labarta, Jesús

doi:10.1109/ipdps.2012.58

Cited by 128 publications

(135 citation statements)

References 17 publications

Supporting

Mentioning

126

Contrasting

Order By: Relevance

“…However, there has also been some efforts to apply task-based approaches to clusters of computers. Cilk-NOW is a variant of Cilk for networks of workstations [8], StarPU-MPI is an extension of StarPU for clusters of acceleratorenhanced machines [9], and OmpSs has been implemented for clusters of GPUs as well [5]. The DAGuE framework [10] is an example of a task-based approach that achieves high performance in dense linear algebra operations.…”

Section: Parallel Programming Modelsmentioning

confidence: 99%

Chunks and Tasks: A programming model for parallelization of dynamic algorithms

Rubensson

Rudberg

2014

Parallel Computing

View full text Add to dashboard Cite

We propose Chunks and Tasks, a parallel programming model built on abstractions for both data and work. The application programmer specifies how data and work can be split into smaller pieces, chunks and tasks, respectively. The Chunks and Tasks library maps the chunks and tasks to physical resources. In this way we seek to combine user friendliness with high performance. An application programmer can express a parallel algorithm using a few simple building blocks, defining data and work objects and their relationships. No explicit communication calls are needed; the distribution of both work and data is handled by the Chunks and Tasks library. This makes efficient implementation of complex applications that require dynamic distribution of work and data easier. At the same time, Chunks and Tasks imposes restrictions on data access and task dependencies that facilitates the development of high performance parallel back ends. We discuss the fundamental abstractions underlying the programming model, as well as performance and fault resilience considerations. We also present a pilot C++ library implementation for clusters of multicore machines and demonstrate its performance for sparse blocked matrix-matrix multiplication.Keywords: distributed memory parallelization, dynamic distribution, dynamic load balancing, fault tolerance, parallel programming model, work stealing Submitted manuscriptThis manuscript was submitted to Parallel Computing (Elsevier) for the special issue devoted to the PMAA 2012 conference.

show abstract

Section: Parallel Programming Modelsmentioning

confidence: 99%

Chunks and Tasks: A programming model for parallelization of dynamic algorithms

Rubensson

Rudberg

2014

Parallel Computing

View full text Add to dashboard Cite

show abstract

“…The most mature of these are StarPU [12,13] and OmpSs [28,37], both of which are block task schedulers. As a class, block task schedulers allow users to specify that a given function should be run asynchronously as a task.…”

Section: Heterogeneous Programming Modelsmentioning

confidence: 99%

“…Task-associative models: Gaining popularity with the rise of task-parallel models and accelerators, this group includes models like OmpSs [27,28,37] and StarPU [12]. Rather than binding data to devices (or using hints and migration), task-associative models define each unit of work as a "task" with an explicit set of input and output data regions on which to operate.…”

Section: Memory Association and Distributionmentioning

confidence: 99%

Runtime Adaptation for Autonomic Heterogeneous Computing

Scogland

Feng

2014

2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

View full text Add to dashboard Cite

Heterogeneity is increasing across all levels of computing, with the rise of accelerators such as GPUs, FPGAs, and other coprocessors into everything from cell phones to supercomputers. More quietly it is increasing with the rise of NUMA systems, hierarchical caching, OS noise, and a myriad of other factors. As heterogeneity becomes a fact of life, efficiently managing heterogeneous compute resources is becoming a critical, and ever more complex, task. The focus of this dissertation is to lay the foundation for an autonomic system for heterogeneous computing, employing runtime adaptation to improve performance portability and performance consistency while maintaining or increasing programmability. We investigate heterogeneity arising from a myriad of factors, grouped into the dimensions of locality and capability. This work has resulted in runtime schedulers capable of automatically detecting and mitigating heterogeneity in physically homogeneous systems through MPI and adaptive coscheduling for physically heterogeneous accelerator based systems as well as a synthesis of the two to address multiple levels of heterogeneity as a coherent whole. We also discuss our current work towards the next generation of fine-grained scheduling and synchronization across heterogeneous platforms in the design of a highly-scalable and portable concurrent queue for many-core systems. Each component addresses aspects of the urgent need for automated management of the extreme and ever expanding complexity introduced by heterogeneity. I have also had the good fortune to collaborate with both Lawrence Livermore National Laboratory and Argonne National Laboratory. Experiences as an intern at each lab has had a significant effect on the final shape of my dissertation. Special thanks go to Dr. Pavan Balaji (again), Dr. Bronis de Supinski (again) and Dr. Barry Rountree. These collaborations proved to be major turning points for me, both in my research and my life as a whole.

show abstract

“…Based on ROSE, Mint translates annotated C code into CUDA code. OmpSs [12,13] is another interesting effort that allows users to define data dependences among tasks. The solution includes a powerful runtime that manages data and schedules tasks among different types of hardware devices, thus requiring little compiler support.…”

Section: Related Workmentioning

confidence: 99%

Early Experiences with the OpenMP Accelerator Model

Liao

Yan

Supinski

et al. 2013

OpenMP in the Era of Low Power Devices and Accelerators

View full text Add to dashboard Cite

Abstract. A recent trend in mainstream computer nodes is the combined use of general-purpose multicore processors and specialized accelerators such as GPUs and DSPs in order to achieve better performance and to reduce power consumption. To support this trend, the OpenMP Language Committee has approved a set of extensions to OpenMP (referred to as the OpenMP accelerator model). The initial version is the subject of Technical Report 1 (TR1) while OpenMP 4.0 Release Candidate 2 (RC2) further refines the extensions. In this paper, we examine the newly released accelerator directives and create an initial reference implementation, referred to as HOMP (Heterogeneous OpenMP). Focused on targeting NVIDIA GPUs, our work is based on an existing OpenMP implementation in the ROSE sourceto-source compiler infrastructure. HOMP includes extensions to parse the new constructs and to represent them in the AST and other compiler translation details. Further we provide initial runtime support. For our evaluation, we have adapted a few existing OpenMP codes to use the accelerator model directives and present preliminary performance results. Finally, we critique the accelerator model in terms of its impact on developers and compiler writers and suggest possible improvements.

show abstract

Productive Programming of GPU Clusters with OmpSs

Cited by 128 publications

References 17 publications

Chunks and Tasks: A programming model for parallelization of dynamic algorithms

Chunks and Tasks: A programming model for parallelization of dynamic algorithms

Runtime Adaptation for Autonomic Heterogeneous Computing

Early Experiences with the OpenMP Accelerator Model

Contact Info

Product

Resources

About