Microarchitectural mechanisms to exploit value structure in SIMT architectures

Abstract

SIMT architectures improve performance and efficiency by exploiting control and memory-access structure across data-parallel threads. Value structure occurs when multiple threads operate on values that can be compactly encoded, e.g., by using a simple function of the thread index. We characterize the availability of control, memory-access, and value structure in typical kernels and observe ample amounts of value structure that is largely ignored by current SIMT architectures. We propose three microarchitectural mechanisms to exploit value structure based on compact affine execution of arithmetic, branch, and memory instructions. We explore these mechanisms within the context of traditional SIMT microarchitectures (GP-SIMT), found in general-purpose graphics processing units, as well as fine-grain SIMT microarchitectures (FG-SIMT), a SIMT variant appropriate for compute-focused data-parallel accelerators. Cycle-level modeling of a modern GP-SIMT system and a VLSI implementation of an eight-lane FG-SIMT execution engine are used to evaluate a range of application kernels. When compared to a baseline without compact affine execution, our approach can improve GP-SIMT cycle-level performance by 4-17% and can improve FG-SIMT absolute performance by 20-65% and energy efficiency up to 30% for a majority of the kernels.

Zeroploit

“…Several researchers have proposed microarchitectural techniques to take advantage of value locality by reusing results from a result cache [54,60] and designing value predictors to enable speculative execution [30,45,46]. Researchers have studied value locality or redundancy in GPGPU compute kernels to detect scalar variables [9,31,42], affine computations [42], and value similarity [63] across threads of SIMT warps, thread-block (TB) dimensionality dependent redundancy across threads of TBs [64], and have proposed hardware-only or hardware-software techniques to gainfully exploit such value locality.…”

Warped-compression

“…Scalar register file [23] eliminates redundant power consumption in that case by storing the thread register value of only one SIMT lane shared across all lanes. Prior works [16,17,30,33] have also used the notion of value similarity. Value structure [16] is exploited for improving memory storage efficiency [17] and power efficient computations [30].…”

“…Prior works [16,17,30,33] have also used the notion of value similarity. Value structure [16] is exploited for improving memory storage efficiency [17] and power efficient computations [30]. However, they did not exploit value structure for improving register file efficiency, which is the primary focus of this paper.…”

“…Prior works [16,17,30,33] have also used the notion of value similarity. The authors in [16] defined the notion of value structure to capture scenarios where different threads within a warp execute on input operands that can be described as a simple function of thread ID and block ID.…”

“…Value structure can then be exploited for improving memory storage efficiency [17]. The authors in [30] exploited this value structure further to create a separate affine functional unit that performs more power efficient computations when a warp instruction exhibits value structure. They also proposed hardware support for branch instruction execution in the presence of value structure to reduce the cost of branch resolution latency.…”