Scaling large-data computations on multi-GPU accelerators

Abstract

Modern supercomputers rely on accelerators to speed up highly parallel workloads. Intricate programming models, limited device memory sizes and overheads of data transfers between CPU and accelerator memories are among the open challenges that restrict the widespread use of accelerators. First, this paper proposes a mechanism and an implementation to automatically pipeline the CPU-GPU memory channel so as to overlap the GPU computation with the memory copies, alleviating the data transfer overhead. Second, in doing so, the paper presents a technique called Computation Splitting, COSP, that caters to arbitrary device memory sizes and automatically manages to run outof-card OpenMP-like applications on GPUs. Third, a novel adaptive runtime tuning mechanism is proposed to automatically select the pipeline stage size so as to gain the best possible performance. The mechanism adapts to the underlying hardware in the starting phase of a program and chooses the pipeline stage size. The techniques are implemented in a system that is able to translate an input OpenMP program to multiple GPUs attached to the same host CPU. Experimentation on a set of nine benchmarks shows that, on average, the pipelining scheme improves the performance by 1.49x, while limiting the runtime tuning overhead to 3% of the execution time.

GPU Accelerated Path Tracing of Massive Scenes

“…UM simplifies both out-of-core processing between GPUs and CPUs as well as multi-GPU processing, 1 and combinations of both. Previously, the applications focusing on large data processing on GPUs required algorithm-specific techniques for memory handling [Al-Saber and Kulkarni 2015;Gelado et al 2010b;Huynh et al 2012;Jablin et al 2012b;Krizhevsky et al 2012;Sabne et al 2013;Seo et al 2015;Shamoto et al 2015].…”

Pagoda

“…GPU researchers have exploited pipelining [29] to overlap data transfers with kernel computations. The distinguishing factor in the Pagoda pipelined task processing is that it overlaps spawning, which comprises the CPU finding a free task entry and performing a data copy, with GPU scheduling, which is only a sub-part of the overall task processing.…”

A scalable approach to solving dense linear algebra problems on hybrid CPU‐GPU systems

“…The generated pipelined code can automatically support computations with out‐of‐GPU datasets. SuperMatrix is another runtime system that supports shared‐memory systems with multiple GPUs . It uses several software cache schemes to maintain the coherence between the host RAM and the GPU memories to minimize communication.…”

“…StarPU relies on a virtual shared memory to handle data transfers and reduce communications. Eigenmann et al [25] proposed a new technique called computation splitting and used the pipelining technique to translate OpenMP programs to run on a host system attached with multiple GPUs. The generated pipelined code can automatically support computations with out-of-GPU datasets.…”