On-the-fly elimination of dynamic irregularities for GPU computing

Abstract

The power-efficient massively parallel Graphics Processing Units (GPUs) have become increasingly influential for general-purpose computing over the past few years. However, their efficiency is sensitive to dynamic irregular memory references and control flows in an application. Experiments have shown great performance gains when these irregularities are removed. But it remains an open question how to achieve those gains through software approaches on modern GPUs. This paper presents a systematic exploration to tackle dynamic irregularities in both control flows and memory references. It reveals some properties of dynamic irregularities in both control flows and memory references, their interactions, and their relations with program data and threads. It describes several heuristics-based algorithms and runtime adaptation techniques for effectively removing dynamic irregularities through data reordering and job swapping. It presents a framework, G-Streamline, as a unified software solution to dynamic irregularities in GPU computing. G-Streamline has several distinctive properties. It is a pure software solution and works on the fly, requiring no hardware extensions or offline profiling. It treats both types of irregularities at the same time in a holistic fashion, maximizing the whole-program performance by resolving conflicts among optimizations. Its optimization overhead is largely transparent to GPU kernel executions, jeopardizing no basic efficiency of the GPU application. Finally, it is robust to the presence of various complexities in GPU applications. Experiments show that G-Streamline is effective in reducing dynamic irregularities in GPU computing, producing speedups between 1.07 and 2.5 for a variety of applications.

BigKernel -- High Performance CPU-GPU Communication Pipelining for Big Data-Style Applications

“…Finally, Zhang et al optimize the layout of irregularly accessed data to achieve more efficient GPU memory accesses by having the CPU place elements of an irregularly accessed array that will be accessed by GPU threads at the same time next to each other, resulting in a higher degree of coalesced accesses [20].…”

Deriving a Methodology for Code Deployment on Multi-Core Platforms via Iterative Manual Optimizations

“…Another performance impacting feature of the Racah program is the degree of warp divergence [34], which can be attributed to the calculation of factorials by means of a loop. To eliminate this overhead, the factorials are pre-computed and stored in texture memory.…”

“…The use of the if-statement and its potential to cause warp divergence is often discussed in the literature [34]. Ironically, the same cannot be said for the logical Boolean operators && and || despite the fact that they too can cause such phenomenon to occur.…”

High Performance Datacenter Networks: Architectures, Algorithms, and Opportunities

“…Zhang et al reduce divergent branches by remapping data locations [105]. Han and Adelrahman propose compiler solutions [46].…”