Performance evaluation of cache replacement policies for the SPEC CPU2000 benchmark suite

Abstract

Replacement policy, one of the key factors determining the effectiveness of a cache, becomes even more important with latest technological trends toward highly associative caches. The state-of-the-art processors employ various policies such as Random, Least Recently Used (LRU), Round-Robin, and PLRU (Pseudo LRU), indicating that there is no common wisdom about the best one. Optimal yet unattainable policy would replace cache memory block whose next reference is the farthest away in the future, among all memory blocks present in the set.In our quest for replacement policy as close to optimal as possible, we thoroughly explored the design space of existing replacement mechanisms using SimpleScalar toolset and SPEC CPU2000 benchmark suite, across wide range of cache sizes and organizations. In order to better understand the behavior of different policies, we introduced new measures, such as cumulative distribution of cache hits in the LRU stack. We also dynamically monitored the number of cache misses, per each 100000 instructions.Our results show that the PLRU techniques can approximate and even outperform LRU with much lower complexity, for a wide range of cache organizations. However, a relatively large gap between LRU and optimal replacement policy, of up to 50%, indicates that new research aimed to close the gap is necessary. The cumulative distribution of cache hits in the LRU stack indicates a very good potential for way prediction using LRU information, since the percentage of hits to the bottom of the LRU stack is relatively high.

Key-Study to Execute Code Using Demand Paging and NAND Flash at Smart Card Scale

“…Most of the time, solutions aim to improve or adapt the replacement policy needed by the demand-paging strategy because researches [18] showed that a gap always exists up to the theoretical best one, known as MIN [19]. When a cache is full and when the CPU tries to read instructions not already present in cache, the new page loaded from the NAND must overwrite an existing one in the cache.…”

Estimating the Performance of Cache Replacement Policies

“…The first source of inaccuracy can be completely eliminated if one is willing to spend additional space to store one stack histogram per cache set. But this is unlikely to allow for great improvements in the precision of predicted miss ratios [1]. The second source has a greater impact on the quality of predictions, but is more difficult to eliminate.…”

“…its state space and transitions. First we need [3,4,1,0] hist: [5] [0, 4 FIFO, history length 1, cutoff age 5) showing some transitions and how probabilities are assigned to them. The value above an arrow is the transition probability.…”

“…Table 1 shows the main results for an 8-way associative cache of size 512kB. 1 We decided to give the bare numbers in this form since the contained information cannot be compactly represented graphically. Only giving average values would conceal the (in)accuracy of individual estimations.…”

“…The replacement policy can have a significant influence on the cache performance [1]. The previous approaches [14,8,9,5] for estimating miss ratios assumed a fixed replacement policy (LRU or random).…”