Decoupled compressed cache

Abstract

In multicore processor systems, last-level caches (LLCs) play a crucial role in reducing system energy by i) filtering out expensive accesses to main memory and ii) reducing the time spent executing in high-power states. Cache compression can increase effective cache capacity and reduce misses, improve performance, and potentially reduce system energy. However, previous compressed cache designs have demonstrated only limited benefits due to internal fragmentation and limited tags.In this paper, we propose the Decoupled Compressed Cache (DCC), which exploits spatial locality to improve both the performance and energy-efficiency of cache compression. DCC uses decoupled super-blocks and non-contiguous sub-block allocation to decrease tag overhead without increasing internal fragmentation. Non-contiguous sub-blocks also eliminate the need for energy-expensive re-compaction when a block's size changes. Compared to earlier compressed caches, DCC increases normalized effective capacity to a maximum of 4 and an average of 2.2 for a wide range of workloads. A further optimized Co-DCC (Co-Compacted DCC) design improves the average normalized effective capacity to 2.6 by co-compacting the compressed blocks in a super-block. Our simulations show that DCC nearly doubles the benefits of previous compressed caches with similar area overhead. We also demonstrate a practical DCC design based on a recent commercial LLC design.

Understanding Cache Compression

“…Compressed caches that decouple tag and data are typically subject to at least one level of indirection, and the ideas usually derive from similar schemes conceived for conventional caches, such as the Indirect-index Cache (IIC) [31] and the Decoupled Sectored Cache [81]. The main difference between decoupling in uncompressed and compressed caches is the granularity: Since compressed blocks can fit in spaces smaller than the size of data entries, such entries can be partitioned into multiple small segments [5,12,32,67,77] (more on segments in Section 4.4).…”

“…Increasing the number of tags has an exceedingly high cost: the tag storage overhead is increased manyfold. Besides, the usefulness of the extra tags is directly proportional to the workload's compressibility; thus, some approaches group multiple tags into a single tag entry, notably reducing the number of tag bits [1,[75][76][77]. This can generally be achieved through the exploitation of two remarks: Neighbor values tend to exhibit approximate similarity, and workloads tend to manifest high spatial locality [75][76][77].…”

“…Finally, the second remark states that neighbor blocks are usually brought into the cache within a small time interval. This means that neighboring blocks are usually located in the cache simultaneously, and their compressibility will likely be similar [77].…”

“…This can be achieved through the addition of a field containing the integral (i.e., unconstrained) size [5,21]-e.g., in a cache with 64-byte cache lines this adds up to 7 bits per tag-or pointers to indicate where the block is placed. Pointers can either associate the segments to their tag [29,77] or the tag to its segments [29,32]. Additionally, the use of pointers allows relaxing the contiguous property: Multiple pointers can be added to specify the location of the dispersed segments of the compressed blocks, removing the need to recompact data [32,77].…”

MBZip

“…Sardashti et al [42] propose a decoupled compressed cache (DCC) that exploits spatial locality to increase the effective cache capacity. DCC uses super block tags to reduce tag overhead.…”