Probabilistic Prediction of Temporal Locality

Abstract

Abstract-The increasing gap between processor and memory speeds, as well as the introduction of multi-core CPUs, have exacerbated the dependency of CPU performance on the memory subsystem. This trend motivates the search for more efficient caching mechanisms, enabling both faster service of frequently used blocks and decreased power consumption. In this paper we describe a novel, random sampling based predictor that can distinguish transient cache insertions from non-transient ones. We show that this predictor can identify a small set of data cache resident blocks that service most of the memory references, thus serving as a building block for new cache designs and block replacement policies. Although we only discuss the L1 data cache, we have found this predictor to be efficient also when handling L1 instruction caches and shared L2 caches.

Workload Modeling for Computer Systems Performance Evaluation

“…In memory reference streams some data blocks are much more popular than others -so much so, that sampling a block at random will produce a block that is seldom used, but sampling a reference at random will most probably identify a block that is highly used [216,217,218]. This can be used to selectively insert only highly used blocks into the cache, without any need to maintain historical information about access patterns to each block.…”

“…It has therefore been suggested that one should focus on the "core" working set, defined to be those pages that are accessed repeatedly a large number of times. The size of this select group is, of course, smaller, and may be claimed to better reflect the locality of the reference stream [447,216,218].…”

“…Although the notion of a set does not imply any order or structure on the memory locations, Ferrari has proposed a generative model that is designed to emulate working sets [257]. There has also been work on combining working sets with skewed popularity, leading to the identification of a core subset of the working set that attracts the majority of references [217,216,218].…”

“…Phases may actually be nested, with a major phase of the computation including several subphases, which may also be repeated [455,410]. In addition, one must distinguish between the reuse set, which is those elements that are indeed reused extensively (i, lenA, and lenB in the earlier example), and the other elements, which are transient in nature [216,410]. The reuse set provides a better approximation of the actual memory capacity needed by the application, because the memory used to store transient elements can be reused.…”

Core Working Set Based Scratchpad Memory Management

“…Denning was the first to formalized the notion of a working set [4] to depict such data items that are accessed within a certain number of instructions. The core working set (CWS) [5] extends Denning's working set concept and illustrates that a dramatic disparity exists between the usage patterns of frequently used data and those of lightly used data in the working set. It is similar to a scenario in SPM management, in which the most popular data items are expected to be maintained in SPM for future references.…”