dOCAL: high-level distributed programming with OpenCL and CUDA

Rasch, Ari; Bigge, Julian; Wrodarczyk, Martin; Schulze, Richard; Gorlatch, Sergei

doi:10.1007/s11227-019-02829-2

Cited by 10 publications

(2 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Strengert et al [28] propose an interesting extension to the CUDA model that extends CUDA's three-level parallelism hierarchy (thread, block, grid) with additional levels (bus, network, application levels). For OpenCL, there have also been several attempts to provide access to remote devices, for example, SnuCL/SnuCL-D [18,20], Distributed OpenCL (dOpenCL) [17], clOpenCL [3], LibWater [13], HybridOpenCL [4], EngineCL [24], and dOCAL [27].…”

Section: Explicit Control Over Multiple/remote Gpusmentioning

confidence: 99%

Lightning: Scaling the GPU Programming Model Beyond a Single GPU

Heldens¹,

Hijma²,

Werkhoven³

et al. 2022

Preprint

View full text Add to dashboard Cite

The GPU programming model is primarily aimed at the development of applications that run one GPU. However, this limits the scalability of GPU code to the capabilities of a single GPU in terms of compute power and memory capacity. To scale GPU applications further, a great engineering effort is typically required: work and data must be divided over multiple GPUs by hand, possibly in multiple nodes, and data must be manually spilled from GPU memory to higher-level memories.We present Lightning: a framework that follows the common GPU programming paradigm but enables scaling to large problems with ease. Lightning supports multi-GPU execution of GPU kernels, even across multiple nodes, and seamlessly spills data to higher-level memories (main memory and disk). Existing CUDA kernels can easily be adapted for use in Lightning, with data access annotations on these kernels allowing Lightning to infer their data requirements and the dependencies between subsequent kernel launches. Lightning efficiently distributes the work/data across GPUs and maximizes efficiency by overlapping scheduling, data movement, and kernel execution when possible.We present the design and implementation of Lightning, as well as experimental results on up to 32 GPUs for eight benchmarks and one realworld application. Evaluation shows excellent performance and scalability, such as a speedup of 57.2× over the CPU using Lighting with 16 GPUs over 4 nodes and 80 GB of data, far beyond the memory capacity of one GPU.

show abstract

Section: Explicit Control Over Multiple/remote Gpusmentioning

confidence: 99%

Lightning: Scaling the GPU Programming Model Beyond a Single GPU

Heldens¹,

Hijma²,

Werkhoven³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Additional work has addressed the issue of expanding OpenCL platforms to clusters that utilise language extensions to facilitate MPI communication and Rasch et al (2019). It provides a SnuCL framework that allows programmers to see all cluster devices as if they're in the host node.…”

Section: Literature Reviewmentioning

confidence: 99%

Adaptive Task Partitioning for Performance Evaluation in Cluster based Heterogeneous Environments

Anitha¹

2021

revistageintec

View full text Add to dashboard Cite

Due to continual server replacement, datacenter-scale clusters are developing toward heterogeneous hardware designs. Meanwhile, datacenters are frequently used by a variety of users for a variety of purposes. Due to multi-tenant interferences, it frequently exhibits high performance heterogeneity. When contrasted to in-house dedicated clusters, deploying MapReduce on such heterogeneous clusters poses major hurdles in attaining adequate application performance. Heterogeneity can cause significant performance degradation in job execution, despite current optimizations on task scheduling and load balancing, because most MapReduce implementations were developed for homogeneous contexts. To make scheduling decisions, the majority of extant adaptive strategies assume a priori knowledge of particular job characteristics. However, without spending a significant cost, such information is not readily available. The suggested framework Adaptive Control Self-tuning provides a significant improvement over existing methods at moderate to high system utilizations, according to the evaluation results.

show abstract