Effective cache bank placement for GPUs

Sadrosadati, Mohammad; Mirhosseini, Amirhossein; Roozkhosh, Shahin; Bakhishi, Hazhir; Sarbazi-Azad, Hamid

doi:10.23919/date.2017.7926954

Cited by 9 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different warps can displace each other's registers in the cache due to the high warp switching rate in GPUs. This thrashing effect is also observed in SMs' local data caches [6,7,8,9,19,24,29,30,33,34,38,42,49,62,64,67,68,75,82,83]. 2.…”

Section: Register File Cachingmentioning

confidence: 61%

LTRF

Sadrosadati

Mirhosseini

Ehsani

et al. 2018

Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Syste

Self Cite

View full text Add to dashboard Cite

Graphics Processing Units (GPUs) employ large register files to accommodate all active threads and accelerate context switching. Unfortunately, register files are a scalability bottleneck for future GPUs due to long access latency, high power consumption, and large silicon area provisioning. Prior work proposes hierarchical register file, to reduce the register file power consumption by caching registers in a smaller register file cache. Unfortunately, this approach does not improve register access latency due to the low hit rate in the register file cache. In this paper, we propose the Latency-Tolerant Register File (LTRF) architecture to achieve low latency in a two-level hierarchical structure while keeping power consumption low. We observe that compile-time interval analysis enables us to divide GPU program execution into intervals with an accurate estimate of a warp's aggregate register working-set within each interval. The key idea of LTRF is to prefetch the estimated register working-set from the main register file to the register file cache under software control, at the beginning of each interval, and overlap the prefetch latency with the execution of other warps. Our experimental results show that LTRF enables high-capacity yet long-latency main GPU register files, paving the way for various optimizations. As an example optimization, we implement the main register file with emerging high-density high-latency memory technologies, enabling 8× larger capacity and improving overall GPU performance by 31% while reducing register file power consumption by 46%.

show abstract

Section: Register File Cachingmentioning

confidence: 61%

LTRF

Sadrosadati

Mirhosseini

Ehsani

et al. 2018

Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Syste

Self Cite

View full text Add to dashboard Cite

show abstract

“…Hence, stencil codes can share input both within and across streaming multiprocessors. In general, the shared data within a thread block can be re-read through shared memory (see blue lines in Figure 2), while the shared data across thread blocks need to be re-read from global memory in GPUs [42,58] (see red lines in Figure 2).…”

Section: Introductionmentioning

confidence: 99%

Efficient Nearest-Neighbor Data Sharing in GPUs

Nematollahi

Sadrosadati

Falahati

et al. 2020

ACM Trans. Archit. Code Optim.

Self Cite

View full text Add to dashboard Cite

Stencil codes (a.k.a. nearest-neighbor computations) are widely used in image processing, machine learning, and scientific applications. Stencil codes incur nearest-neighbor data exchange because the value of each point in the structured grid is calculated as a function of its value and the values of a subset of its nearest-neighbor points. When running on Graphics Processing Unit (GPUs), stencil codes exhibit a high degree of data sharing between nearest-neighbor threads. Sharing is typically implemented through shared memories, shuffle instructions, and on-chip caches and often incurs performance overheads due to the redundancy in memory accesses. In this article, we propose Neighbor Data (NeDa), a direct nearest-neighbor data sharing mechanism that uses two registers embedded in each streaming processor (SP) that can be accessed by nearest-neighbor SP cores. The registers are compiler-allocated and serve as a data exchange mechanism to eliminate nearest-neighbor shared accesses. NeDa is embedded carefully with local wires between SP cores so as to minimize the impact on density. We place and route NeDa in an open-source GPU and show a small area overhead of 1.3%. The cycle-accurate simulation indicates an average performance improvement of 21.8% and power reduction of up to 18.3% for stencil codes in General-Purpose Graphics Processing Unit (GPGPU) standard benchmark suites. We show that NeDa’s performance is within 13.2% of an ideal GPU with no overhead for nearest-neighbor data exchange.

show abstract