A Hierarchical Thread Scheduler and Register File for Energy-Efficient Throughput Processors

Abstract

Modern graphics processing units (GPUs) employ a large number of hardware threads to hide both function unit and memory access latency. Extreme multithreading requires a complex thread scheduler as well as a large register file, which is expensive to access both in terms of energy and latency. We present two complementary techniques for reducing energy on massively-threaded processors such as GPUs. First, we investigate a two-level thread scheduler that maintains a small set of active threads to hide ALU and local memory access latency and a larger set of pending threads to hide main memory latency. Reducing the number of threads that the scheduler must consider each cycle improves the scheduler's energy efficiency. Second, we propose replacing the monolithic register file found on modern designs with a hierarchical register file. We explore various tradeoffs for the hierarchy including the number of levels in the hierarchy and the number of entries at each level. We consider both a hardware-managed caching scheme and a softwaremanaged scheme, where the compiler is responsible for orchestrating all data movement within the register file hierarchy. Combined with a hierarchical register file, our two-level thread scheduler provides a further reduction in energy by only allocating entries in the upper levels of the register file hierarchy for active threads. Averaging across a variety of real world graphics and compute workloads, the active thread count can be reduced by a factor of 4 with minimal impact on performance and our most efficient three-level software-managed register file hierarchy reduces register file energy by 54%.

Unified on-chip memory allocation for SIMT architecture

“…While this work focused on modeling of concurrent execution, it did not discuss how to achieve the desired concurrency level. Other relevant GPU work includes topics such as GPU exception handling [18], where register states need to be restored for resuming execution after exception, and energy saving [10], where the location of registers is critical to energy consumption because the distance between the registers and processors determines the amount of energy consumed during data movement, and hardware register space saving [29],which combines SRAM and DRAM to store more bits into the die area. In [28], a means of optimizing shared memory is explored in order to prevent userallocated shared memory from reducing occupancy, whereas our approach makes use of non user-allocated shared memory to lessen the cost of improving occupancy.…”

Skewed Multistaged Multibanked Register File for Area and Energy Efficiency

“…This is the latest proposal on the register file for area and energy efficiency [20], and researchers in NVIDIA adopted this idea for their GPUs [21], [22].…”

A Stall-Aware Warp Scheduling for Dynamically Optimizing Thread-level Parallelism in GPGPUs

“…The commonly used Loose Round Robin (LRR) [17] serves all warps as instruction issuance candidates to maximize TLP. Previous work [16,30] has showed that LRR cannot efficiently hide the long latency operations, and causes high resource contention.…”

“…Existing warp scheduling schemes such as LRR [17] and TL [16,30] try either to maximize TLP or to select a universal TLP parameter. Previous work [16,23] shows that the maximized or a universal TLP does not always deliver the optimal performance, because it is impossible to fit various access patterns for diverse applications.…”