Reuse distance-based shared LLC management mechanism for heterogeneous CPU-GPU systems

Abstract

Heterogeneous systems with CPU and GPU cores integrated on a single chip provide fast communication between CPU and GPU cores and high performance. The last-level cache (LLC) is one of the most essential shared memory resources in heterogeneous CPU-GPU systems. Due to the different LLC access behaviors and collaboration patterns from CPU and GPU, the shared LLC is not guaranteed to be used efficiently without any management. In this paper, we propose a reuse distance-based LLC management mechanism to improve system performance and save energy consumption. Our mechanism monitors reuse distances of cache lines accessed by CPU and GPU in sampling cache sets. It compares average reuse distance values of CPU and GPU cores to determine cache line insertion, promotion, and eviction in the LLC. Our mechanism can achieve significant performance improvement with a higher LLC hit rate. The evaluation results show that our management method can improve performance by up to 16% and 7%, and achieve LLC energy savings of up to 13% and 4% over the LRU and TAP-RRIP policies, respectively.