ChargeCache: Reducing DRAM latency by exploiting row access locality

Hassan, Hasan; Pekhimenko, Gennady; Vijaykumar, Nandita; Seshadri, Vivek; Lee, Donghyuk; Ergin, Oğuz; Mutlu, Onur

doi:10.1109/hpca.2016.7446096

Cited by 132 publications

(117 citation statements)

References 75 publications

Supporting

Mentioning

115

Contrasting

Order By: Relevance

“…Third, works that aim to reduce DRAM latency by modifying the microarchitecture of DRAM or the memory controller [31,56,57,91,92,108,157,168]. These works reduce latency without introducing bit errors.…”

Section: Related Workmentioning

confidence: 99%

Eden

Koppula

Orosa

Yağlıkçı

et al. 2019

Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture

Self Cite

View full text Add to dashboard Cite

The effectiveness of deep neural networks (DNN) in vision, speech, and language processing has prompted a tremendous demand for energy-efficient high-performance DNN inference systems. Due to the increasing memory intensity of most DNN workloads, main memory can dominate the system's energy consumption and stall time. One effective way to reduce the energy consumption and increase the performance of DNN inference systems is by using approximate memory, which operates with reduced supply voltage and reduced access latency parameters that violate standard specifications. Using approximate memory reduces reliability, leading to higher bit error rates. Fortunately, neural networks have an intrinsic capacity to tolerate increased bit errors. This can enable energy-efficient and high-performance neural network inference using approximate DRAM devices.Based on this observation, we propose EDEN, the first general framework that reduces DNN energy consumption and DNN evaluation latency by using approximate DRAM devices, while strictly meeting a user-specified target DNN accuracy. EDEN relies on two key ideas: 1) retraining the DNN for a target approximate DRAM device to increase the DNN's error tolerance, and 2) efficient mapping of the error tolerance of each individual DNN data type to a corresponding approximate DRAM partition in a way that meets the user-specified DNN accuracy requirements.We evaluate EDEN on multi-core CPUs, GPUs, and DNN accelerators with error models obtained from real approximate DRAM devices. We show that EDEN's DNN retraining technique reliably improves the error resiliency of the DNN by an order of magnitude. For a target accuracy within 1% of the original DNN, our results show that EDEN enables 1) an average DRAM energy reduction of 21%, 37%, 31%, and 32% in CPU, GPU, and two different DNN accelerator architectures, respectively, across a variety of state-ofthe-art networks, and 2) an average (maximum) speedup of 8% (17%) and 2.7% (5.5%) in CPU and GPU architectures, respectively, when evaluating latency-bound neural networks.

show abstract

Section: Related Workmentioning

confidence: 99%

Eden

Koppula

Orosa

Yağlıkçı

et al. 2019

Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture

Self Cite

View full text Add to dashboard Cite

show abstract

“…To fully utilize the available memory bandwidth, memory controllers would have to maximize the number of or commands. However, due to interference between access streams within and across applications, memory controllers need to constantly open and close rows by issuing and commands due to an increasing number of bank con icts [44,68]. These commands increase the queuing latency of accesses ( and ), thus decreasing the e ective memory bandwidth utilization.…”

Section: Long-term Impact On Industrymentioning

confidence: 99%

“…We refer the reader to our prior works[22,24,25,26,43,44,61,64,65,66,67,68,76,77,79,81,82,86,87,110,127,128] for a detailed background on DRAM.…”

mentioning

confidence: 99%

Understanding Latency Variation in Modern DRAM Chips

Chang

Kashyap

Hassan

et al. 2016

SIGMETRICS Perform. Eval. Rev.

Self Cite

View full text Add to dashboard Cite

This article summarizes key results of our work on experimental characterization and analysis of latency variation and latency-reliability trade-o s in modern DRAM chips, which was published in SIGMETRICS 2016 [24], and examines the work's signi cance and future potential. Our work is motivated to reduce the long DRAM latency, which is a critical performance bottleneck in current systems. DRAM access latency is de ned by three fundamental operations that take place within the DRAM cell array: (i) activation of a memory row, which opens the row to perform accesses; (ii) precharge, which prepares the cell array for the next memory access; and (iii) restoration of the row, which restores the values of cells in the row that were destroyed due to activation. There is signi cant latency variation for each of these operations across the cells of a single DRAM chip due to irregularity in the manufacturing process. As a result, some cells are inherently faster to access, while others are inherently slower. Unfortunately, existing systems do not exploit this variation.

show abstract

“…We demonstrate 28% average performance improvement in Section 6.1 for our evaluated workloads. As core count increases, bank conflicts also increase and become a bigger performance bottleneck [22,23,28].…”

Section: Introductionmentioning

confidence: 99%

Enabling and Exploiting Partition-Level Parallelism (PALP) in Phase Change Memories

Song

Das

Mutlu

et al. 2019

ACM Trans. Embed. Comput. Syst.

Self Cite

View full text Add to dashboard Cite

Phase-change memory (PCM) devices have multiple banks to serve memory requests in parallel. Unfortunately, if two requests go to the same bank, they have to be served one after another, leading to lower system performance. We observe that a modern PCM bank is implemented as a collection of partitions that operate mostly independently while sharing a few global peripheral structures, which include the sense amplifiers (to read) and the write drivers (to write). Based on this observation, we propose PALP, a new mechanism that enables partition-level parallelism within each PCM bank, and exploits such parallelism by using the memory controller's access scheduling decisions. PALP consists of three new contributions. First, we introduce new PCM commands to enable parallelism in a bank's partitions in order to resolve the read-write bank conflicts, with no changes needed to PCM logic or its interface. Second, we propose simple circuit modifications that introduce a new operating mode for the write drivers, in addition to their default mode of serving write requests. When configured in this new mode, the write drivers can resolve the read-read bank conflicts, working jointly with the sense amplifiers. Finally, we propose a new access scheduling mechanism in PCM that improves performance by prioritizing those requests that exploit partition-level parallelism over other requests, including the long outstanding ones. While doing so, the memory controller also guarantees starvation-freedom and the PCM's running-average-power-limit (RAPL).We evaluate PALP with workloads from the MiBench and SPEC CPU2017 Benchmark suites. Our results show that PALP reduces average PCM access latency by 23%, and improves average system performance by 28% compared to the state-of-the-art approaches. CCS CONCEPTS• Computer systems organization → Architectures; • Hardware → Memory and dense storage.

show abstract

ChargeCache: Reducing DRAM latency by exploiting row access locality

Cited by 132 publications

References 75 publications

Eden

Eden

Understanding Latency Variation in Modern DRAM Chips

Enabling and Exploiting Partition-Level Parallelism (PALP) in Phase Change Memories

Contact Info

Product

Resources

About