Improving GPU performance via large warps and two-level warp scheduling

Abstract

Due to their massive computational power, graphics processing units (GPUs) have become a popular platform for executing general purpose parallel applications. GPU programming models allow the programmer to create thousands of threads, each executing the same computing kernel. GPUs exploit this parallelism in two ways. First, threads are grouped into fixed-size SIMD batches known as warps, and second, many such warps are concurrently executed on a single GPU core. Despite these techniques, the computational resources on GPU cores are still underutilized, resulting in performance far short of what could be delivered. Two reasons for this are conditional branch instructions and stalls due to long latency operations.To improve GPU performance, computational resources must be more effectively utilized. To accomplish this, we propose two independent ideas: the large warp microarchitecture and two-level warp scheduling. We show that when combined, our mechanisms improve performance by 19.1% over traditional GPU cores for a wide variety of general purpose parallel applications that heretofore have not been able to fully exploit the available resources of the GPU chip.

Toggle-Aware Compression for GPUs

“…Several recent works focus on bandwidth compression to decrease memory traffic by transmitting data in a compressed form in both CPUs [17], [24], [3] and GPUs [21], [17], [26], which results in better system performance and energy consumption. Bandwidth compression proves to be particularly effective in GPUs because GPUs are often bottlenecked by memory bandwidth [15], [14], [13], [28], [26]. GPU applications also exhibit high degrees of data redundancy [21], [17], [26], leading to good compression ratios.…”

Warp-level divergence in GPUs: Characterization, impact, and mitigation

“…Compared to the resources shown in Table 1, the hardware overhead of our proposed approach, including the TB dispatcher logic and the per-SM 40-bit workload buffer, is nearly negligible. The baseline warp scheduling policy is round robin (RR) and the two-level warp scheduling policy [17] is examined in our design space exploration in Section 6.3. In our design space exploration, we also vary the register file size and the SIMD width to evaluate the effectiveness of our approach in different configurations.…”

A Study of the Potential of Locality-Aware Thread Scheduling for GPUs

“…[6,7]. Recent work has focussed on two-level warp scheduling to reduce the impact of memory latency [4,8]. Although we not address control flow, we note that an ideal scheduler takes both aspects (data-locality and control flow) into account.…”