The dynamic granularity memory system

Yoon, Kyoungro; Jeong, Jeong; Sullivan,; Erez,

doi:10.1109/isca.2012.6237047

Cited by 45 publications

(58 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Leveraging the observation that applications access only a few words within a cache block, researchers have proposed re-engineering the processor/memory interface to allow for activating only the DRAM chips at which the requested words are stored [1,16,58,59]. While potentially effective in the server context as well, these proposals require disruptive changes to commodity memory technology.…”

Section: Related Workmentioning

confidence: 99%

“…Instruction-based predictors have been extensively used in the context of data prefetching [25,44], cache-coherence action prediction [19], NOC power reduction [21], last-write prediction [50], on-chip granularity prediction [26], and offchip bandwidth reduction [17,58]. However, none of these works has targeted improving row buffer locality by predicting the memory page access density.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

BuMP: Bulk Memory Access Prediction and Streaming

Volos¹,

Picorel²,

Falsafi³

et al. 2014

2014 47th Annual IEEE/ACM International Symposium on Microarchitecture

View full text Add to dashboard Cite

Abstract-With the end of Dennard scaling, server power has emerged as the limiting factor in the quest for more capable datacenters. Without the benefit of supply voltage scaling, it is essential to lower the energy per operation to improve server efficiency. As the industry moves to lean-core server processors, the energy bottleneck is shifting toward main memory as a chief source of server energy consumption in modern datacenters. Maximizing the energy efficiency of today's DRAM chips and interfaces requires amortizing the costly DRAM page activations over multiple row buffer accesses.This work introduces Bulk Memory Access Prediction and Streaming, or BuMP. We make the observation that a significant fraction (59-79%) of all memory accesses fall into DRAM pages with high access density, meaning that the majority of their cache blocks will be accessed within a modest time frame of the first access. Accesses to high-density DRAM pages include not only memory reads in response to load instructions, but also reads stemming from store instructions as well as memory writes upon a dirty LLC eviction. The remaining accesses go to low-density pages and virtually unpredictable reference patterns (e.g., hashed key lookups). BuMP employs a low-cost predictor to identify high-density pages and triggers bulk transfer operations upon the first read or write to the page. In doing so, BuMP enforces high row buffer locality where it is profitable, thereby reducing DRAM energy per access by 23%, and improves server throughput by 11% across a wide range of server applications.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

BuMP: Bulk Memory Access Prediction and Streaming

Volos¹,

Picorel²,

Falsafi³

et al. 2014

2014 47th Annual IEEE/ACM International Symposium on Microarchitecture

View full text Add to dashboard Cite

show abstract

“…Repetitive calls to these functions result in repetitive data access patterns (i.e., page footprints) that can be exploited to predict future data accesses upon subsequent calls to the same function. The correlation between code and data access patterns has been heavily exploited for data prefetching [2], [15], [27] and filtering of unused data [10], [14], [16], [32].…”

Section: A Unison Cachementioning

confidence: 99%

Unison Cache: A Scalable and Effective Die-Stacked DRAM Cache

Jevdjic¹,

Loh

Kaynak³

et al. 2014

2014 47th Annual IEEE/ACM International Symposium on Microarchitecture

142

113

View full text Add to dashboard Cite

Abstract-Recent research advocates large die-stacked DRAM caches in manycore servers to break the memory latency and bandwidth wall. To realize their full potential, diestacked DRAM caches necessitate low lookup latencies, high hit rates and the efficient use of off-chip bandwidth. Today's stacked DRAM cache designs fall into two categories based on the granularity at which they manage data: block-based and page-based. The state-of-the-art block-based design, called Alloy Cache, colocates a tag with each data block (e.g., 64B) in the stacked DRAM to provide fast access to data in a single DRAM access. However, such a design suffers from low hit rates due to poor temporal locality in the DRAM cache. In contrast, the state-of-the-art page-based design, called Footprint Cache, organizes the DRAM cache at page granularity (e.g., 4KB), but fetches only the blocks that will likely be touched within a page. In doing so, the Footprint Cache achieves high hit rates with moderate on-chip tag storage and reasonable lookup latency. However, multi-gigabyte stacked DRAM caches will soon be practical and needed by server applications, thereby mandating tens of MBs of tag storage even for page-based DRAM caches.We introduce a novel stacked-DRAM cache design, Unison Cache. Similar to Alloy Cache's approach, Unison Cache incorporates the tag metadata directly into the stacked DRAM to enable scalability to arbitrary stacked-DRAM capacities. Then, leveraging the insights from the Footprint Cache design, Unison Cache employs large, page-sized cache allocation units to achieve high hit rates and reduction in tag overheads, while predicting and fetching only the useful blocks within each page to minimize the off-chip traffic. Our evaluation using server workloads and caches of up to 8GB reveals that Unison cache improves performance by 14% compared to Alloy Cache due to its high hit rate, while outperforming the state-of-the art page-based designs that require impractical SRAM-based tags of around 50MB.

show abstract

“…Fortunately, coherence snoops are not common in many applications (e.g., 1/100 cache operations in SpecJBB) as a coherence directory and an inclusive LLC filter them out. 1 We do not have access to an industry-grade 32nm library, so we synthesized at a higher 180nm node size and scaled the results to 32 nm (latency and energy scaled proportional to Vdd (taken from [36]) and V dd 2 respectively). Large caches with many words per set (≡ highly associative conventional cache) need careful consideration.…”

Section: Tag-only Operationsmentioning

confidence: 99%

“…[7]) to restructure data for improved spatial efficiency. There have also been efforts from the architecture community to predict spatial locality [29,34,17,36], which we can leverage to predict Amoeba-Block ranges. Finally, cache compression is an orthogonal body of work that does not eliminate unused words but seeks to minimize the overall memory footprint [1].…”

Section: Related Workmentioning

confidence: 99%

Amoeba-Cache: Adaptive Blocks for Eliminating Waste in the Memory Hierarchy

Kumar

Zhao

Shriraman

et al. 2012

2012 45th Annual IEEE/ACM International Symposium on Microarchitecture

View full text Add to dashboard Cite

The fixed geometries of current cache designs do not adapt to the working set requirements of modern applications, causing significant inefficiency. The short block lifetimes and moderate spatial locality exhibited by many applications result in only a few words in the block being touched prior to eviction. Unused words occupy between 17-80% of a 64K L1 cache and between 1%-79% of a 1MB private LLC. This effectively shrinks the cache size, increases miss rate, and wastes on-chip bandwidth. Scaling limitations of wires mean that unused-word transfers comprise a large fraction (11%) of on-chip cache hierarchy energy consumption.We propose Amoeba-Cache, a design that supports a variable number of cache blocks, each of a different granularity. Amoeba-Cache employs a novel organization that completely eliminates the tag array, treating the storage array as uniform and morphable between tags and data. This enables the cache to harvest space from unused words in blocks for additional tag storage, thereby supporting a variable number of tags (and correspondingly, blocks). Amoeba-Cache adjusts individual cache line granularities according to the spatial locality in the application. It adapts to the appropriate granularity both for different data objects in an application as well as for different phases of access to the same data. Overall, compared to a fixed granularity cache, the Amoeba-Cache reduces miss rate on average (geometric mean) by 18% at the L1 level and by 18% at the L2 level and reduces L1-L2 miss bandwidth by 46%. Correspondingly, Amoeba-Cache reduces on-chip memory hierarchy energy by as much as 36% (mcf) and improves performance by as much as 50% (art).

show abstract

The dynamic granularity memory system

Cited by 45 publications

References 24 publications

BuMP: Bulk Memory Access Prediction and Streaming

BuMP: Bulk Memory Access Prediction and Streaming

Unison Cache: A Scalable and Effective Die-Stacked DRAM Cache

Amoeba-Cache: Adaptive Blocks for Eliminating Waste in the Memory Hierarchy

Contact Info

Product

Resources

About