Design guidelines for high-performance SCM hierarchies

Abstract

With emerging storage-class memory (SCM) nearing commercialization, there is evidence that it will deliver the much-anticipated high density and access latencies within only a few factors of DRAM. Nevertheless, the latency-sensitive nature of memory-resident services makes seamless integration of SCM in servers questionable. In this paper, we ask the question of how best to introduce SCM for such servers to improve overall performance/cost over existing DRAM-only architectures. We first show that even with the most optimistic latency projections for SCM, the higher memory access latency results in prohibitive performance degradation. However, we find that deployment of a modestly sized high-bandwidth 3D stacked DRAM cache makes the performance of an SCM-mostly memory system competitive. The high degree of spatial locality that memory-resident services exhibit not only simplifies the DRAM cache's design as page-based, but also enables the amortization of increased SCM access latencies and the mitigation of SCM's read/write latency disparity.We identify the set of memory hierarchy design parameters that plays a key role in the performance and cost of a memory system combining an SCM technology and a 3D stacked DRAM cache. We then introduce a methodology to drive provisioning for each of these design parameters under a target performance/cost goal. Finally, we use our methodology to derive concrete results for specific SCM technologies. With PCM as a case study, we show that a two bits/cell technology hits the performance/cost sweet spot, reducing the memory subsystem cost by 40% while keeping performance within 3% of the best performing DRAM-only system, whereas single-level and triple-level cell organizations are impractical for use as memory replacements.• Information systems → Storage class memory; Cloud based storage; • Hardware → Memory and dense storage; KEYWORDS Storage-class memory, heterogeneous memory hierarchy, 3D stacked DRAM ACM Reference Format:

Tearing Down the Memory Wall

“…However, such a scheme suffers from long latencies and low throughput due to high overhead from the software stack. Prior work such as [18,20,45] has tackled this high overhead by enabling cache line access to the SSD [18,20], using host DRAM as a cache [45], merging multiple translation layers into one layer, and promoting pages from SSD when locality is detected [18]. Due to its benefits, Flash memory is abundantly available, extensively used, and continues to be optimized to provide better performance [13,22,35].…”

AstriFlash A Flash-Based System for Online Services

“…First, many modern online services have msscale end-to-end tail-latency constraints [18], [23], which allows them to absorb few µs-scale flash accesses [14], [18], [22], [41], [42], [44]. Second, object popularity and request distributions for datacenter workloads are inherently skewed [64], [73], [75], [76], thus allowing hosting the hot fraction of the dataset in DRAM that serves most requests and filters the bandwidth required from the backing flash. The above observations should permit the design of a cost-effective two-tier hierarchy where a capacity-constrained DRAM caches the hot fraction of the dataset stored in a capacity-scaled flash layer.…”

Virtualization techniques for memory resource exploitation

“…This approach allows for the memory to be scaled out independently of the processing resources, because scaling up the memory within a node is becoming increasingly prohibitive [108]. This "share something" approach for remote memory targets some of the challenges that are very relevant in today's data centers, some of which that are:…”

“…When memory is disaggregated and becomes a pooled resource, it is possible to use different types of memory technologies besides the traditional DRAM, such as Storage-Class Memories (SCMs) [108], a term that refers generally to Non-Volatile Memories (NVMs) [77]. NVMs provide a good option for pooled disaggregated memory because it is able to provide higher density and higher capacity than traditional DRAMs, but at a much lower cost [28].…”

“…NVMs provide a good option for pooled disaggregated memory because it is able to provide higher density and higher capacity than traditional DRAMs, but at a much lower cost [28]. However, NVMs have higher latency than DRAM (but still less latency than Flash or disk devices), less bandwidth and has endurance limitations [108,126,28]. NVMs can be used as an additional level in the memory hierarchy functioning as a cache between DRAM and permanent storage (even in cases when it is not pooled) [108,110], as block-device for swapped pages [126,40], or for increasing system resiliency through data replication [122] or as back-up for system memory [96].…”