Reshaping cache misses to improve row-buffer locality in multicore systems

Abstract

General-purpose graphics processing units (GPG-PUs) are at their best in accelerating computation by exploiting abundant thread-level parallelism (TLP) offered by many classes of HPC applications. To facilitate such high TLP, emerging programming models like CUDA and OpenCL allow programmers to create work abstractions in terms of smaller work units, called cooperative thread arrays (CTAs). CTAs are groups of threads and can be executed in any order, thereby providing ample opportunities for TLP. The state-of-the-art GPGPU schedulers allocate maximum possible CTAs per-core (limited by available on-chip resources) to enhance performance by exploiting TLP.However, we demonstrate in this paper that executing the maximum possible number of CTAs on a core is not always the optimal choice from the performance perspective. High number of concurrently executing threads might cause more memory requests to be issued, and create contention in the caches, network and memory, leading to long stalls at the cores. To reduce resource contention, we propose a dynamic CTA scheduling mechanism, called DYNCTA, which modulates the TLP by allocating optimal number of CTAs, based on application characteristics. To minimize resource contention, DYNCTA allocates fewer CTAs for applications suffering from high contention in the memory subsystem, compared to applications demonstrating high throughput. Simulation results on a 30-core GPGPU platform with 31 applications show that the proposed CTA scheduler provides 28% average improvement in performance compared to the existing CTA scheduler.

GPU Performance vs. Thread-Level Parallelism

“…Prior studies on the impact of TLP on throughput-oriented processors, including References [14,18,27], are discussed in Section 4.…”

“…The inclusion of multiple levels of caches complicates the relationship between TLP and the overall performance. As studied in prior works [14,18,27], high degrees of TLP cause the cache to suffer from the contention problem, which may lead to performance degradation. For example, Figure 1 shows the impact of cache thrashing with various degrees of TLP.…”

CaLRS: A Critical‐Aware Shared LLC Request Scheduling Algorithm on GPGPU

“…Chen et al [18] proposed a novel warp scheduling algorithm that flexibly uses the time slice round-robin feature to utilize GPU parallelism. Jog et al [19] and Kayiran et al [20] proposed CTA-aware warp scheduling algorithms to reduce cache and memory contention or improve thread-level parallelism. Rogers et al [4] analyzed how hardware scheduler influences the management to GPU cache and proposed a cache sensitive warp scheduling policy.…”