A Stall-Aware Warp Scheduling for Dynamically Optimizing Thread-level Parallelism in GPGPUs

Abstract

General-Purpose Graphic Processing Units (GPGPU) have been widely used in high performance computing as application accelerators due to their massive parallelism and high throughput. A GPGPU generally contains two layers of schedulers, a cooperative-thread-array (CTA) scheduler and a warp scheduler, which administer the thread level parallelism (TLP). Previous research shows the maximized TLP does not always deliver the optimal performance. Unfortunately, existing warp scheduling schemes do not optimize TLP at runtime, which is impossible to fit various access patterns for diverse applications. Dynamic TLP optimization in the warp scheduler remains a challenge to exploit the GPGPU highly-parallel compute power.In this paper, we comprehensively investigate the TLP performance impact in the warp scheduler. Based on our analysis of the pipeline efficiency, we propose a Stall-Aware Warp Scheduling (SAWS), which optimizes the TLP according to the pipeline stalls. SAWS adds two modules to the original scheduler to dynamically adjust TLP at runtime. A trigger-based method is employed for a fast tuning response. We simulated SAWS and conducted extensive experiments on GPGPU-Sim using 21 paradigmatic benchmarks. Our numerical results show that SAWS effectively improves the pipeline efficiency by reducing the structural hazards without causing extra data hazards. SAWS achieves an average speedup of 14.7% with a geometric mean, even higher than existing Two-Level scheduling scheme with the optimal fetch group sizes over a wide range of benchmarks. More importantly, compared with the dynamic TLP optimization in the CTA scheduling, SAWS still has 9.3% performance improvement among the benchmarks, which shows that it is a competitive choice by moving dynamic TLP optimization from the CTA to warp scheduler.

Improving Thread-level Parallelism in GPUs Through Expanding Register File to Scratchpad Memory

“…EXPARS can improve TLP by enabling more CTAs per SM through expanding register file to scratchpad memory. However, previous works [5,13,19,43] have shown that higher TLP does not always mean higher performance due to resource contention. To alleviate the contention, we propose a Lazy Two-Level Warp Scheduler (LTLWS), which is inspired by Reference [19], to control the maximum number of schedulable warps (active warps) during runtime.…”

“…Jing et al [12] introduce an integrated architecture that enables the register file to support the function of cache, which also has above weaknesses. GPU warp scheduling: GPU warp scheduling is a hot research point in recent years [19,20,31,36,43]. Lee et al [20] first propose a profiling algorithm to find the critical warps and then schedule these critical warps more frequently than others.…”

Haws

“…In addition, Kayiran et al [11] proposed a dynamic CTA scheduling technique that attempts to allocate optimal number of CTAs per core based on application demands, demonstrating that executing the maximum number of CTAs per core is not always the best solution to boost performance due to high cache and memory contention. Yu et al [33] presented a Stall-Aware Warp Scheduling (SAWS) policy, which dynamically optimizes the TLP according to pipeline stalls. SAWS can effectively improve pipeline efficiency by reducing structural hazards without introducing new data hazards.…”

Memory-aware circuit overlay NoCs for latency optimized GPGPU architectures

“…Recent research on GPGPUs has led to the optimization of thread level parallelism and maximizing the execution of cooperative thread arrays [1][2][3]. This has made GPGPUs more viable for high performance computation.…”