The ZCache: Decoupling Ways and Associativity

Abstract

Abstract-The ever-increasing importance of main memory latency and bandwidth is pushing CMPs towards caches with higher capacity and associativity. Associativity is typically improved by increasing the number of ways. This reduces conflict misses, but increases hit latency and energy, placing a stringent trade-off on cache design. We present the zcache, a cache design that allows much higher associativity than the number of physical ways (e.g. a 64-associative cache with 4 ways). The zcache draws on previous research on skew-associative caches and cuckoo hashing. Hits, the common case, require a single lookup, incurring the latency and energy costs of a cache with a very low number of ways. On a miss, additional tag lookups happen off the critical path, yielding an arbitrarily large number of replacement candidates for the incoming block.Unlike conventional designs, the zcache provides associativity by increasing the number of replacement candidates, but not the number of cache ways. To understand the implications of this approach, we develop a general analysis framework that allows to compare associativity across different cache designs (e.g. a set-associative cache and a zcache) by representing associativity as a probability distribution. We use this framework to show that for zcaches, associativity depends only on the number of replacement candidates, and is independent of other factors (such as the number of cache ways or the workload). We also show that, for the same number of replacement candidates, the associativity of a zcache is superior than that of a set-associative cache for most workloads. Finally, we perform detailed simulations of multithreaded and multiprogrammed workloads on a large-scale CMP with zcache as the last-level cache. We show that zcaches provide higher performance and better energy efficiency than conventional caches without incurring the overheads of designs with a large number of ways.

Last level cache partitioning via multiverse thread classification

“…Vantage assumes a special cache organization (ZCache) is in place [14]. The TMD strategy is based on a custom design of the allocation policy so that the allocation and enforcement policies work in harmony.…”

Ubik

“…It is simple, but it supports a small number of coarsely-sized partitions (in multiples of way size); partition associativity is proportional to its way count, so partitioning degrades performance; and more importantly, reconfigurations are slow and unpredictable (Section 7). Alternatively, Vantage [45] leverages the statistical properties of skew-associative caches [48] and zcaches [44] to implement partitioning efficiently. Vantage supports fine-grained partitions (defined in cache lines), provides strict guarantees on partition sizes and isolation, can resize partitions without moves or invalidations, reconfiguration transients are much faster than in way-partitioning [45], and it is cheap to implement (requiring ≈1% extra state and negligible logic).…”

“…Every miss in that partition grows its size by one line, evicting lines from partitions that are being downsized (through Vantage's two-stage demotion-eviction process [45]). Vantage leverages the statistical properties of zcaches [44] to guarantee that a growing partition has a negligible probability of suffering an eviction (about once in a million accesses) independently of the access pattern, so we can safely assume that no line is evicted until the partition reaches its target 2 . Under these conditions, transients are easy to predict.…”

Arbitrary Modulus Indexing

“…More sophisticated approaches use multiple bit-hash schemes, either on multiple ways, such as skewed-associative caches [9], or on the same way, such as column-associative caches [17], hashrehash caches [18], ZCaches [19], [20], or various indirect indexing schemes [21], [8]. Many of these schemes are difficult to implement in practice.…”