Efficient footprint caching for Tagless DRAM Caches

Jang, Hakbeom; Lee, Yong-Jun; Kim, Jong-Won; Kim, Youngsok; Kim, Jangwoo; Jeong, Jinkyu; Lee, Jae Woo

doi:10.1109/hpca.2016.7446068

Cited by 38 publications

(35 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An alternate approach to designing DRAM caches is to use large granularity caches to reduce tag and metadata overhead, in hardware [14,53] or software [54,55,56,57]. The reduction in tag requirements enable more space for asso-ciativity and replacement metadata.…”

Section: Page-based Dram Cachesmentioning

confidence: 99%

“…The reduction in tag requirements enable more space for asso-ciativity and replacement metadata. Such large-granularity caches often employ recency-based replacement [14,54,55] or frequency-based replacement [56,57] that would otherwise be too expensive in line-granularity caches. We compare with Unison cache [14] (hardware-managed, 4-way, page-based sectored cache with LRU replacement, separate tag lookup) as a representative of page-based designs, in Figure 23.…”

Section: Page-based Dram Cachesmentioning

confidence: 99%

See 1 more Smart Citation

To Update or Not To Update?: Bandwidth-Efficient Intelligent Replacement Policies for DRAM Caches

Young

Qureshi

2019

2019 IEEE 37th International Conference on Computer Design (ICCD)

View full text Add to dashboard Cite

This paper investigates intelligent replacement policies for improving the hit-rate of gigascale DRAM caches. Cache replacement policies are commonly used to improve the hit-rate of on-chip caches. The most effective replacement policies often require the cache to track and update per-line reuse state to inform their decision. A fundamental challenge on DRAM caches, however, is that stateful policies would require significant bandwidth to maintain per-line DRAM cache state. As such, DRAM cache replacement policies have primarily been stateless policies, such as always-install or probabilistic bypass. Unfortunately, we find that stateless policies are often too coarse-grain and become ineffective at the size and associativity of DRAM caches. Ideally, we want a replacement policy that can obtain the hit-rate benefits of stateful replacement policies, but keep the bandwidth-efficiency of stateless policies.We perform our study on a DRAM cache design similar to the one used in Knights Landing, and find that tracking per-line reuse state can enable an effective replacement policy that can mitigate the common thrashing patterns seen in gigascale caches. We propose a stateful replacement/bypass policy called RRIP Age-On-Bypass (RRIP-AOB), that tracks reuse state for high-reuse lines, protects such lines by bypassing other lines, and Ages the state On cache Bypass. Unfortunately, such a stateful technique requires significant bandwidth to update state. To this end, we propose Efficient Tracking of Reuse (ETR). ETR makes state tracking efficient by accurately tracking the state of only one line from a region, and using the state of that line to guide the replacement decisions for other lines in that region. ETR reduces the bandwidth for tracking the replacement state by 70%, and makes stateful policies practical for DRAM caches. Our evaluations with a 2GB DRAM cache, show that our RRIP-AOB and ETR techniques provide 18% speedup while needing less than 1KB of SRAM. DRAM ARRAY DATA (64B)ECC-bits (8B) ROW BUFFERRow Buffer = 32 x (72 byte TAD) AddressData Burst (4 x 18B)

show abstract

Section: Page-based Dram Cachesmentioning

confidence: 99%

Section: Page-based Dram Cachesmentioning

confidence: 99%

To Update or Not To Update?: Bandwidth-Efficient Intelligent Replacement Policies for DRAM Caches

Young

Qureshi

2019

2019 IEEE 37th International Conference on Computer Design (ICCD)

View full text Add to dashboard Cite

show abstract

“…An alternate approach to designing DRAM-caches is to use large-granularity caches to amortize tag and metadata overhead, in hardware [10,13] or software [35,36,37,38].…”

Section: Page-based Dram Cachesmentioning

confidence: 99%

“…Software-supported: Software-supported DRAM cache approaches maintain mapping and metadata information inside page tables [35,36,37,38], and use various heuristics to determine when to install pages. The benefit to such approaches is that they do not need to pay additional bandwidth to access tags.…”

Section: Page-based Dram Cachesmentioning

confidence: 99%

TicToc: Enabling Bandwidth-Efficient DRAM Caching for Both Hits and Misses in Hybrid Memory Systems

Young

Chishti

Qureshi

2019

2019 IEEE 37th International Conference on Computer Design (ICCD)

View full text Add to dashboard Cite

This paper investigates bandwidth-efficient DRAM caching for hybrid DRAM + 3D-XPoint memories. 3D-XPoint is becoming a viable alternative to DRAM as it enables highcapacity and non-volatile main memory systems. However, 3D-XPoint has several characteristics that limit it from outright replacing DRAM: 4-8x slower read, and even worse writes. As such, effective DRAM caching in front of 3D-XPoint is important to enable a high-capacity, low-latency, and high-write-bandwidth memory. There are currently two major approaches for DRAM cache design: (1) a Tag-Inside-Cacheline (TIC) organization that optimizes for hits, by storing tag next to each line such that one access gets both tag and data, and (2) a Tag-Outside-Cacheline (TOC) organization that optimizes for misses, by storing tags from multiple data lines together in a tag-line such that one access to a tag-line gets information on several data-lines. Ideally, we would like to have the low hit-latency of TIC designs, and the low miss-bandwidth of TOC designs. To this end, we propose a TicToc organization that provisions both TIC and TOC to get the hit and miss benefits of both.We find that naively combining both techniques actually performs worse than TIC individually, because one has to pay the bandwidth cost of maintaining both metadata. The main contribution of this work is developing architectural techniques to reduce bandwidth cost of accessing and maintaining both TIC and TOC metadata. We find that most of the update bandwidth is due to maintaining the TOC dirty information. We propose a DRAM Cache Dirtiness Bit technique that carries DRAM cache dirty information to last-level caches, to help prune repeated dirty-bit updates for known dirty lines. We also propose a Preemptive Dirty Marking technique that predicts which lines will be written and proactively marks the dirty bit at install time, to help avoid the initial dirty-bit update for dirty lines. To support PDM, we develop a novel PC-based Write-Predictor to aid in marking only write-likely lines. Our evaluations on a 4GB DRAM cache in front of 3D-XPoint show that our TicToc organization enables 10% speedup over the baseline TIC, nearing the 14% speedup possible with an idealized DRAM cache design with 64MB of SRAM tags, while needing only 34KB SRAM.

show abstract

“…Metadata overhead: The metadata requirement for DRAMCaches, even for caches of size 64MB, is large and is in the order of few MBs. The large storage requirement along with the associated cost if it has to be stored in SRAM, has driven DRAMCache designs to either use larger block sizes (of size 2KB or 4KB [26,28]) to reduce metadata overhead or co-locate metadata alongside data in the DRAMCache [25,34,39].…”

Section: Hashcache Designmentioning

confidence: 99%

HAShCache

Patil

Govindarajan

2017

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

Integrated Heterogeneous System (IHS) processors pack throughput-oriented General-Purpose Graphics Pprocessing Units (GPGPUs) alongside latency-oriented Central Processing Units (CPUs) on the same die sharing certain resources, e.g., shared last-level cache, Network-on-Chip (NoC), and the main memory. The demands for memory accesses and other shared resources from GPU cores can exceed that of CPU cores by two to three orders of magnitude. This disparity poses significant problems in exploiting the full potential of these architectures. In this article, we propose adding a large-capacity stacked DRAM, used as a shared last-level cache, for the IHS processors. However, adding the DRAMCache naively, leaves significant performance on the table due to the disparate demands from CPU and GPU cores for DRAMCache and memory accesses. In particular, the imbalance can significantly reduce the performance benefits that the CPU cores would have otherwise enjoyed with the introduction of the DRAMCache, necessitating a heterogeneity-aware management of this shared resource for improved performance. In this article, we propose three simple techniques to enhance the performance of CPU application while ensuring very little to no performance impact to the GPU. Specifically, we propose (i) PrIS, a prioritization scheme for scheduling CPU requests at the DRAMCache controller; (ii) ByE, a selective and temporal bypassing scheme for CPU requests at the DRAMCache; and (iii) Chaining, an occupancy controlling mechanism for GPU lines in the DRAMCache through pseudo-associativity. The resulting cache, Heterogeneity-Aware Shared DRAMCache (HAShCache), is heterogeneity-aware and can adapt dynamically to address the inherent disparity of demands in an IHS architecture. Experimental evaluation of the proposed HAShCache results in an average system performance improvement of 41% over a naive DRAMCache and over 200% improvement over a baseline system with no stacked DRAMCache. CCS Concepts: • Computer systems organization → Heterogeneous (hybrid) systems; • Hardware → 3D integrated circuits; Dynamic memory; • Theory of computation → Caching and paging algorithms;

show abstract

Efficient footprint caching for Tagless DRAM Caches

Cited by 38 publications

References 25 publications

To Update or Not To Update?: Bandwidth-Efficient Intelligent Replacement Policies for DRAM Caches

To Update or Not To Update?: Bandwidth-Efficient Intelligent Replacement Policies for DRAM Caches

TicToc: Enabling Bandwidth-Efficient DRAM Caching for Both Hits and Misses in Hybrid Memory Systems

HAShCache

Contact Info

Product

Resources

About