Improving GPGPU Performance via Cache Locality Aware Thread Block Scheduling

Chen, Li-Jhan; Cheng, Hsiang-Yun; Wang, Po-Han; Yang, Chia-Lin

doi:10.1109/lca.2017.2693371

Cited by 13 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chen et al [8] propose a hardware-software approach for applications with structural data access, both row-and column-major applications. It checks the address ranges of the ready TBs and issues the TB with the maximum overlapping address range with the TBs already executing on that SM, increasing data reuse and improving the performance.…”

Section: Related Workmentioning

confidence: 99%

Paver

Tripathy

Abdolrashidi

Bhuyan

et al. 2021

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

The massive parallelism present in GPUs comes at the cost of reduced L1 and L2 cache sizes per thread, leading to serious cache contention problems such as thrashing. Hence, the data access locality of an application should be considered during thread scheduling to improve execution time and energy consumption. Recent works have tried to use the locality behavior of regular and structured applications in thread scheduling, but the difficult case of irregular and unstructured parallel applications remains to be explored. We present PAVER , a P riority- A ware V ertex schedul ER , which takes a graph-theoretic approach toward thread scheduling. We analyze the cache locality behavior among thread blocks ( TBs ) through a just-in-time compilation, and represent the problem using a graph representing the TBs and the locality among them. This graph is then partitioned to TB groups that display maximum data sharing, which are then assigned to the same streaming multiprocessor by the locality-aware TB scheduler. Through exhaustive simulation in Fermi, Pascal, and Volta architectures using a number of scheduling techniques, we show that PAVER reduces L2 accesses by 43.3%, 48.5%, and 40.21% and increases the average performance benefit by 29%, 49.1%, and 41.2% for the benchmarks with high inter-TB locality.

show abstract

Section: Related Workmentioning

confidence: 99%

Paver

Tripathy

Abdolrashidi

Bhuyan

et al. 2021

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

show abstract

“…Locality-aware scheduling. To improve cache locality, in these methods, the threads are categorized into groups, at the level of warps [33,43] or thread blocks [7,30,35], and given a priority. When the execution of one group is stalled, the other category is given priority.…”

Section: Related Workmentioning

confidence: 99%

Efficient Nearest-Neighbor Data Sharing in GPUs

Nematollahi

Sadrosadati

Falahati

et al. 2020

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

Stencil codes (a.k.a. nearest-neighbor computations) are widely used in image processing, machine learning, and scientific applications. Stencil codes incur nearest-neighbor data exchange because the value of each point in the structured grid is calculated as a function of its value and the values of a subset of its nearest-neighbor points. When running on Graphics Processing Unit (GPUs), stencil codes exhibit a high degree of data sharing between nearest-neighbor threads. Sharing is typically implemented through shared memories, shuffle instructions, and on-chip caches and often incurs performance overheads due to the redundancy in memory accesses. In this article, we propose Neighbor Data (NeDa), a direct nearest-neighbor data sharing mechanism that uses two registers embedded in each streaming processor (SP) that can be accessed by nearest-neighbor SP cores. The registers are compiler-allocated and serve as a data exchange mechanism to eliminate nearest-neighbor shared accesses. NeDa is embedded carefully with local wires between SP cores so as to minimize the impact on density. We place and route NeDa in an open-source GPU and show a small area overhead of 1.3%. The cycle-accurate simulation indicates an average performance improvement of 21.8% and power reduction of up to 18.3% for stencil codes in General-Purpose Graphics Processing Unit (GPGPU) standard benchmark suites. We show that NeDa’s performance is within 13.2% of an ideal GPU with no overhead for nearest-neighbor data exchange.

show abstract

“…Unfortunately, this exploits locality between consecutive CTAs located in a row only. Chen et al [22] propose a software-hardware cooperative design to exploit spatial locality among different CTAs located in different rows and columns. Li et al [23] propose software techniques to schedule CTAs with potential reuse on the same SM to exploit inter-CTA locality on real GPU hardware.…”

Section: Related Workmentioning

confidence: 99%

Intra-Cluster Coalescing and Distributed-Block Scheduling to Reduce GPU NoC Pressure

Wang

Zhao

Kaeli

et al. 2019

IEEE Trans. Comput.

View full text Add to dashboard Cite

GPUs continue to boost the number of streaming multiprocessors (SMs) to provide increasingly higher compute capabilities. To construct a scalable crossbar network-on-chip (NoC) that connects the SMs to the memory controllers, a cluster structure is introduced in modern GPUs in which several SMs are grouped together to share a network port. Because of network port sharing, clustered GPUs face severe NoC congestion, which creates a critical performance bottleneck. In this paper, we target redundant network traffic to mitigate GPU NoC congestion. In particular, we observe that in many GPU-compute applications, different SMs in a cluster access shared data. Sending redundant requests to access the same memory location wastes valuable NoC bandwidth-we find on average 19% (and up to 48%) of the requests to be redundant. To remove redundant NoC traffic, we propose distributed-block scheduling, intra-cluster coalescing (ICC) and the coalesced cache (CC) to coalesce L1 cache misses within and across SMs in a cluster, respectively. Our evaluation results show that distributed-block scheduling, ICC and CC are complementary and improve both performance and energy consumption. We report an average performance improvement of 15% (and up to 67%) while at the same time reducing system energy by 6% (and up to 19%) and improving the energy-delay product (EDP) by 19% on average (and up to 53%), compared to state-of-the-art distributed CTA scheduling.

show abstract

Improving GPGPU Performance via Cache Locality Aware Thread Block Scheduling

Cited by 13 publications

References 11 publications

Paver

Paver

Efficient Nearest-Neighbor Data Sharing in GPUs

Intra-Cluster Coalescing and Distributed-Block Scheduling to Reduce GPU NoC Pressure

Contact Info

Product

Resources

About