Scalable framework for mapping streaming applications onto multi-GPU systems

GPUs offer an order of magnitude higher compute power and memory bandwidth than CPUs. GPUs therefore might appear to be well suited to accelerate computations that operate on voluminous data sets in independent ways; e.g., for transformations, filtering, aggregation, partitioning or other "Big Data" style processing. Yet experience indicates that it is difficult, and often error-prone, to write GPGPU programs which efficiently process data that does not fit in GPU memory, partly because of the intricacies of GPU hardware architecture and programming models, and partly because of the limited bandwidth available between GPUs and CPUs.In this paper, we propose BigKernel, a scheme that provides pseudo-virtual memory to GPU applications and is implemented using a 4-stage pipeline with automated prefetching to (i) optimize CPU-GPU communication and (ii) optimize GPU memory accesses. BigKernel simplifies the programming model by allowing programmers to write kernels using arbitrarily large data structures that can be partitioned into segments where each segment is operated on independently; these kernels are transformed into BigKernel using straight-forward compiler transformations.Our evaluation on six data-intensive benchmarks shows that BigKernel achieves an average speedup of 1.7 over state-of-the-art double-buffering techniques and an average speedup of 3.0 over corresponding multi-threaded CPU implementations.

show abstract

“…[1], [7], [9]. Interestingly, they also target streaming computations, but they assume the data is already in GPU global memory.…”

Section: Related Workmentioning

confidence: 99%

BigKernel -- High Performance CPU-GPU Communication Pipelining for Big Data-Style Applications

Mokhtari

Stumm

2014

2014 IEEE 28th International Parallel and Distributed Processing Symposium

View full text Add to dashboard Cite

show abstract

“…Beyond the algorithm skeleton realm, the StreamIt [7] language and the NMM [11] multimedia middleware leverage the use of multiple GPUs for stream processing, mainly applied to the multimedia and digital signal processing fields. Mapping stream processing onto GPUs is done by partitioning the graph across the multiple devices, and by streaming the data along this graph in a pipelined fashion, migrating it between GPUs when required, until the end is reached.…”

Section: Related Workmentioning

confidence: 99%

On the support of task-parallel algorithmic skeletons for multi-GPU computing

Alexandre

Marques

Paulino

2014

Proceedings of the 29th Annual ACM Symposium on Applied Computing

View full text Add to dashboard Cite

An emerging trend in the field of Graphics Processing Unit (GPU) computing is the harnessing of multiple devices to cope with scalability and performance requirements. However, multi-GPU execution adds new challenges to the already complex world of General Purpose computing on GPUs (GPGPU), such as the efficient problem decomposition, and dealing with device heterogeneity. To this extent, we propose the use of the Marrow algorithmic skeleton framework (ASkF) to abstract most of the details intrinsic to the programming of such platforms. To the best of our knowledge, Marrow is the first ASkF to support skeleton nesting on single and (now) multiple GPU systems. In this paper we present how it can transparently distribute the execution of skeleton compositions among a set of, possibly, heterogeneous devices. An experimental evaluation assesses the proposal's effectiveness, from a scalability and performance perspective, with good results.

show abstract

“…Secondly, the applicability of the stream programming model to large applications is still an open challenge. Further work by Huynh et al [27] performs multi-level partitioning of the stream graph to overcome the scalability challenges in [26]. In this work, a mechanism to port StreamIt programs to multi-GPUs on the same system has also been proposed.…”

Section: Related Workmentioning

confidence: 99%

Scaling large-data computations on multi-GPU accelerators

Sabne

Sakdhnagool

Eigenmann

2013

Proceedings of the 27th International ACM Conference on International Conference on Supercomputing

View full text Add to dashboard Cite

Modern supercomputers rely on accelerators to speed up highly parallel workloads. Intricate programming models, limited device memory sizes and overheads of data transfers between CPU and accelerator memories are among the open challenges that restrict the widespread use of accelerators. First, this paper proposes a mechanism and an implementation to automatically pipeline the CPU-GPU memory channel so as to overlap the GPU computation with the memory copies, alleviating the data transfer overhead. Second, in doing so, the paper presents a technique called Computation Splitting, COSP, that caters to arbitrary device memory sizes and automatically manages to run outof-card OpenMP-like applications on GPUs. Third, a novel adaptive runtime tuning mechanism is proposed to automatically select the pipeline stage size so as to gain the best possible performance. The mechanism adapts to the underlying hardware in the starting phase of a program and chooses the pipeline stage size. The techniques are implemented in a system that is able to translate an input OpenMP program to multiple GPUs attached to the same host CPU. Experimentation on a set of nine benchmarks shows that, on average, the pipelining scheme improves the performance by 1.49x, while limiting the runtime tuning overhead to 3% of the execution time.

show abstract

Scalable framework for mapping streaming applications onto multi-GPU systems

Cited by 36 publications

References 23 publications

BigKernel -- High Performance CPU-GPU Communication Pipelining for Big Data-Style Applications

BigKernel -- High Performance CPU-GPU Communication Pipelining for Big Data-Style Applications

On the support of task-parallel algorithmic skeletons for multi-GPU computing

Scaling large-data computations on multi-GPU accelerators

Contact Info

Product

Resources

About