Locality-Aware CTA Scheduling for Gaming Applications

Abstract

The compute work rasterizer or the GigaThread Engine of a modern NVIDIA GPU focuses on maximizing compute work occupancy across all streaming multiprocessors in a GPU while retaining design simplicity. In this article, we identify the operational aspects of the GigaThread Engine that help it meet those goals but also lead to less-than-ideal cache locality for texture accesses in 2D compute shaders, which are an important optimization target for gaming applications. We develop three software techniques, namely LargeCTAs , Swizzle , and Agents , to show that it is possible to effectively exploit the texture data working set overlap intrinsic to 2D compute shaders. We evaluate these techniques on gaming applications across two generations of NVIDIA GPUs, RTX 2080 and RTX 3080, and find that they are effective on both GPUs. We find that the bandwidth savings from all our software techniques on RTX 2080 is much higher than the bandwidth savings on baseline execution from inter-generational cache capacity increase going from RTX 2080 to RTX 3080. Our best-performing technique, Agents , records up to a 4.7% average full-frame speedup by reducing bandwidth demand of targeted shaders at the L1-L2 and L2-DRAM interfaces by 23% and 32%, respectively, on the latest generation RTX 3080. These results acutely highlight the sensitivity of cache locality to compute work rasterization order and the importance of locality-aware cooperative thread array scheduling for gaming applications.

Boustrophedonic Frames: Quasi-Optimal L2 Caching for Textures in GPUs

“…Another work [8] proposes a NUCA organization for the L1 texture caches to increase their effective overall capacity. Focusing on texture locality-aware workload scheduling to different shader cores with software modifications, Ukarande et al [48] report a 4% speedup when exploiting Texture Cache locality on high-end desktop graphics workloads. Another work [21] also exploits Texture Cache locality by scheduling quads that are closer in screen coordinates.…”