Vector-aware register allocation for GPU shader processors

You, Yi-Ping; Chen, Szu-Chieh

doi:10.1109/cases.2015.7324550

Cited by 4 publications

(5 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several approaches have been proposed to improve GPU register usage. Many of them exploit smarter register allocation [10,11,30,37,40]. Similar to our work, Hayes and Zhang [11] proposed a register allocation algorithm for GPUs that utilizes the shared memory for spilling.…”

Section: Related Workmentioning

confidence: 78%

See 1 more Smart Citation

RegDem: Increasing GPU Performance via Shared Memory Register Spilling

Sakdhnagool,

Sabne,

Eigenmann

2019

Preprint

View full text Add to dashboard Cite

GPU utilization, measured as occupancy, is limited by the parallel threads' combined usage of on-chip resources, such as registers and the programmer-managed shared memory. Higher resource demand means lower effective parallel thread count, and therefore lower program performance. Our investigation found that registers are often the occupancy limiters.The de-facto nvcc compiler-based approach spills excessive registers to the off-chip memory, ignoring the shared memory and leaving the on-chip resources underutilized. To mitigate the register demand, this paper presents a binary translation technique, called RegDem, that spills excessive registers to the underutilized shared memory by transforming the GPU assembly code (SASS). Most GPU programs do not fully use shared memory, thus allowing RegDem to use it for register spilling. The higher occupancy achieved by RegDem outweighs the slightly higher cost of accessing shared memory instead of placing data in registers. The paper also presents a compile-time performance predictor that models instructions stalls to choose the best version from a set of program variants. Cumulatively, these techniques outperform the nvcc compiler with a 9% geometric mean, the highest observed being 18%.

show abstract

Section: Related Workmentioning

confidence: 78%

“…Prior work has proposed several approaches to reduce register pressure on GPUs [10,11,30,37,40]. The closest related GPU register allocation algorithm [11] operates on the binary generated by nvcc with aggressive register allocation.…”

Section: Introductionmentioning

confidence: 99%

RegDem: Increasing GPU Performance via Shared Memory Register Spilling

Sakdhnagool,

Sabne,

Eigenmann

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…In addition to traditional general‐purpose architecture, register allocation is also a key problem for massively parallel architectures such as GPU. You and Chen 20 proposed a vector‐aware allocation approach to efficiently manage the register files on the shader architecture. Chen et al 21 proposed a register allocation approach for Intel HD and Iris Graphics, which features a large byte‐addressable register file and an expressive instruction set, by decoupling the assignment of local and global variables.…”

Section: Related Workmentioning

confidence: 99%

Space‐address decoupled scratchpad memory management for neural network accelerators

Zhang

Sun

Chen³

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

Deep neural networks have been demonstrated to be useful in varieties of intelligent tasks, and various specialized NN accelerators have been proposed recently to improve the hardware efficiency, which are typically equipped with software-managed scratchpad memory (SPM) for high performance and energy efficiency. However, traditional SPM management techniques cause memory fragmentation for NN accelerators, and thus lead to low utilization of precious SPM. The main reason is that traditional techniques are originally designed for managing fixed-length registers rather than variable-length memory blocks. In this article, we propose a novel SPM management approach for NN accelerators. The basic intuition is that NN computation/memory behaviors are predictable and relatively regular compared with traditional applications, and thus most information can be determined at compile time. In addition, by exploiting the variable-length feature of SPM, we propose to divide the allocation process into two passes: the space assignment and the address assignment pass, which are simultaneously (and implicitly) performed in traditional one-pass allocation techniques. Experimental results on the memory requests of a representative NN accelerator demonstrate that the proposed approach can significantly reduce the memory consumption by 30% at most compared with state-of-the-art SPM management techniques, and the memory usage is only 2% larger than that of the theoretical optimal allocation. K E Y W O R D S deep neural network, memory management, scratchpad memory 1 INTRODUCTION Deep neural networks (DNNs) have been widely used for various applications, such as computer vision, 1 speech recognition, 2 machine translation, 3 and robotics, 4 due to the improved accuracy over traditional machine learning approaches. However, the performance benefits of DNNs come at the cost of extremely high computation and memory complexity, which pose great challenges to underlying hardware architecture. To improve the efficiency of DNN processing, various specialized accelerators have been proposed to deliver orders of magnitude better performance and energy efficiency than general-purpose architectures such as CPUs and GPUs. 5,6 Specialized neural network accelerators typically require various novel architectural components, including control logics (e.g., DianNao 5 employs a control processor with dedicated control instructions and Eyeriss 6 employs two-level control hierarchy 6), computational units (e.g., DianNao employs 16-fix functional units to leverage the error-tolerance features of intelligent applications), and memory hierarchy Zhenxing Zhang and Shiyan Sun contributed equally to this work.

show abstract

“…For YHFT‐DSP, Tang et al 27 studied the allocation algorithm for register pairs. Besides, some people studied register allocation methods for other GPU architectures 28–31 . However, all of the above methods are for the register allocation of general scalar registers and general vector registers.…”

Section: Introductionmentioning

confidence: 99%

“…Besides, some people studied register allocation methods for other GPU architectures. [28][29][30][31] However, all of the above methods are for the register allocation of general scalar registers and general vector registers. In graph coloring register allocation, when the particular roles of some physical registers should be considered, the corresponding interference graph contains register allocation candidates (hereinafter referred to as "allocation candidates" or "candidates") and some register names.…”

Section: Introductionmentioning

confidence: 99%

Research on global register allocation for code containing array‐unit dual‐usage register names

Zhang

Wang

et al. 2022

Concurrency and Computation

View full text Add to dashboard Cite

Summary Array‐unit dual‐usage register is a kind of register resource that can be read or written as a whole or individually. It is mainly configured in processors with SIMD processing units and provides register‐level speed data transfer between the scalar and vector processing units. To improve the efficiency of algorithms by using an array‐unit dual‐usage register, we investigate in this article the problem of adapting register allocation to code containing array‐unit dual‐usage register names. We propose a corresponding global register allocation method by combining the allocation of regular registers with array‐unit dual‐usage register, ensuring that the names of array‐unit dual‐usage register can be used in the input code of register allocation. Moreover, we present the processing framework of this method and the specific algorithms of some related vital aspects and demonstrate the working principles of the algorithms by an example. Experimental studies were conducted on a platform based on the FT‐M7002 DSP core, and showing that our register allocation method can effectively handle codes containing array‐unit dual‐usage register names and support relevant application algorithms to improve their data transfer scheme. For some typical algorithms with input matrix, substantial performance improvements of twofold or higher are achieved.

show abstract

Vector-aware register allocation for GPU shader processors

Cited by 4 publications

References 8 publications

RegDem: Increasing GPU Performance via Shared Memory Register Spilling

RegDem: Increasing GPU Performance via Shared Memory Register Spilling

Space‐address decoupled scratchpad memory management for neural network accelerators

Research on global register allocation for code containing array‐unit dual‐usage register names

Contact Info

Product

Resources

About