Exploiting reuse locality on inclusive shared last-level caches

Abstract

Optimization of the replacement policy used for Shared Last-Level Cache (SLLC) management in a Chip-MultiProcessor (CMP) is critical for avoiding off-chip accesses. Temporal locality, while being exploited by first levels of private cache memories, is only slightly exhibited by the stream of references arriving at the SLLC. Thus, traditional replacement algorithms based on recency are bad choices for governing SLLC replacement. Recent proposals involve SLLC replacement policies that attempt to exploit reuse either by segmenting the replacement list or improving the rereference interval prediction. On the other hand, inclusive SLLCs are commonplace in the CMP market, but the interaction between replacement policy and the enforcement of inclusion has barely been discussed. After analyzing that interaction, this article introduces two simple replacement policies exploiting reuse locality and targeting inclusive SLLCs: Least Recently Reused (LRR) and Not Recently Reused (NRR). NRR has the same implementation cost as NRU, and LRR only adds one bit per line to the LRU cost. After considering reuse locality and its interaction with the invalidations induced by inclusion, the proposals are evaluated by simulating multiprogrammed workloads in an 8-core system with two private cache levels and an SLLC. LRR outperforms LRU by 4.5% (performing better in 97 out of 100 mixes) and NRR outperforms NRU by 4.2% (performing better in 99 out of 100 mixes). We also show that our mechanisms outperform rereference interval prediction, a recently proposed SLLC replacement policy and that similar conclusions can be drawn by varying the associativity or the SLLC size.

Block value based insertion policy for high performance last-level caches

“…If there are no such blocks, the replacement policy will select the victim block from all blocks in the cache set. This method is similar to a recently proposed inclusive cache management policy [1]. To make a fair comparison, we evaluate the performance of all techniques using this extension in our experiments.…”

The reuse cache

“…Despite these efforts, state of the art replacement policies only achieve average improvements within 5% of a commercial algorithm such as NRU, even using a set of benchmarks whose performance is sensitive to replacement [12,1]. Moreover, the fraction of live lines in the SLLC does not increase much with any of the aforementioned techniques.…”

Scrutinizing Resource Utilization for High Performance and Low Energy Computation

“…Recent works [4,5,31,32,34,39,43,56,59,66,73,76] have shown that the traditional LRU replacement policy is highly ineffective for two kinds of workloads. (i) When the working set is larger than the cache size, high-reuse blocks evict each other due to the lack of space, making the cache ineffective.…”

“…Recent works [4,5,31,32,34,39,43,56,59,66,73,76] have shown that the traditional LRU replacement policy is highly ineffective for workloads with cache thrashing and pollution. A large number of works propose insertion policies to solve these problems [16,18,31,32,34,44,56,59,61,66,73,76] by either statistically keeping a fraction of the working set in the cache or exploiting special mechanisms to differentiate the reuse locality of cache blocks and use that information to assist cache block bypassing or insertion.…”

“…However, the LLC is not utilized efficiently. Since the reference stream to the LLC is filtered by the L1 and L2 caches, it has been shown that a large portion of blocks in the LLC are not reused between insertion and eviction if using the traditional least recently used (LRU) replacement policy [4,5,39,43,59].…”