Fundamental limits on the precision of in-memory architectures

Gonugondla, Sujan K.; Sakr, Charbel; Dbouk, Hassan; Shanbhag, Naresh R.

doi:10.1145/3400302.3416344

Cited by 21 publications

(7 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1) Comprehending the fundamental efficiency vs. accuracy trade-offs in IMCs (see [107], [110]), including minimizing the column ADC's energy cost (see [97], [100], [104]). [circuits, statistical analysis] 2) Developing algorithmic approaches such as Shannoninspired statistical error compensation (SEC) [38], [111], [112] and approximate computing [47] to enhance the accuracy of IMCs beyond what is possible via purely 3) Leveraging emerging devices, e.g., MRAM, FeFET, to design IMCs with unique energy, latency, accuracy trade-offs.…”

Section: Summary and Discussionmentioning

confidence: 99%

Benchmarking In-Memory Computing Architectures

Shanbhag

Roy

2022

IEEE Open J. Solid-State Circuits Soc.

Self Cite

View full text Add to dashboard Cite

In-memory computing (IMC) architectures have emerged as a compelling platform to implement energy efficient machine learning (ML) systems. However, today, the energy efficiency gains provided by IMC designs seem to be leveling off and it is not clear what the limiting factors are. The conceptual complexity of IMCs combined with the absence of a rigorous benchmarking methodology makes it difficult to gauge progress and identify bottlenecks in this exciting field. This paper presents a benchmarking methodology for IMCs comprising: 1) a compositional view of IMCs that enables one to parse an IMC design into its canonical components; 2) a set of benchmarking metrics to quantify the performance, efficiency, and accuracy of IMCs; and 3) a strategy for analyzing the reported IMC data and metrics. We apply the proposed benchmarking methodology on an extensive database of IMC metrics extracted from > 70 IC designs published since 2018, in order to infer and comprehend trends in this area.Our benchmarking effort indicates: 1) SRAM-based IMCs show a clear win in terms of energy efficiency and compute density over digital accelerators at the bank-level but the energy efficiency gap reduces dramatically when comparing at the processor level; 2) eNVM-based IMCs lag behind SRAM-based IMCs in terms of both energy efficiency and compute density, and surprisingly lag digital accelerators in terms of compute density; 3) the compute (bank-level) accuracy of IMCs, though a critical metric, is pervasively neglected in publications as is the energy vs. accuracy trade-off inherent to IMCs.

show abstract

Section: Summary and Discussionmentioning

confidence: 99%

Benchmarking In-Memory Computing Architectures

Shanbhag

Roy

2022

IEEE Open J. Solid-State Circuits Soc.

Self Cite

View full text Add to dashboard Cite

show abstract

“…AMS designs are currently under development both in academia [4] and industry [21]. Recent works [6], [22], [23] have sought to characterize the fundamental energy-accuracy tradeoffs with analog multiplication hardware, focusing on the limits imposed by ADC noise.…”

Section: Related Workmentioning

confidence: 99%

“…Ghodrati et al [7] proposed to reduce the effect of ADC noise via parallel bit-interleaved analog compute units sharing a single ADC. Dynamic analog precision has been proposed by Gonugondla et al [22] to adapt to the variation in noise sensitivity across different portions of network architectures. Garg et al [24] proposed averaging the results of multiple matrix multiplications to reduce the effect of analog device noise.…”

Section: Related Workmentioning

confidence: 99%

Adaptive Block Floating-Point for Analog Deep Learning Hardware

Basumallik¹,

Bunandar²,

Dronen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Analog mixed-signal (AMS) devices promise faster, more energy-efficient deep neural network (DNN) inference than their digital counterparts. However, recent studies show that DNNs on AMS devices with fixed-point numbers can incur an accuracy penalty because of precision loss. To mitigate this penalty, we present a novel AMS-compatible adaptive block floating-point (ABFP) number representation. We also introduce amplification (or gain) as a method for increasing the accuracy of the number representation without increasing the bit precision of the output. We evaluate the effectiveness of ABFP on the DNNs in the MLPerf™ datacenter inference benchmark-realizing less than 1% loss in accuracy compared to FLOAT32. We also propose a novel method of finetuning for AMS devices, Differential Noise Finetuning (DNF), which samples device noise to speed up finetuning compared to conventional Quantization-Aware Training.

show abstract

“…The result for SQNR qiy(dB) in ( 8) follows by taking SQNR qiy(dB) = 10 log 10 (SQNR qiy ), with SQNR qiy given by (29). given by (30).…”

Section: Appendix a Sqnr Expressionsmentioning

confidence: 99%

“…A comprehensive analytical understanding of the relationship between precision, compute SNR, energy, and delay across all types of IMCs, is presently missing. This paper fills this gap (preliminary results in [29]) by: 1) defining compute SNR metrics for IMCs, 2) developing a systematic methodology to obtain a minimum precision assignment for activations, weights and outputs of fixed-point DPs realized on IMCs to meet network accuracy requirements, and 3) employing this methodology to obtain the limits on achievable compute SNR of commonly employed IMC topologies, and quantify their energy vs. accuracy tradeoffs.…”

Section: Introductionmentioning

confidence: 99%

Fundamental Limits on Energy-Delay-Accuracy of In-memory Architectures in Inference Applications

Gonugondla¹,

Sakr²,

Dbouk³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

This paper obtains fundamental limits on the computational precision of in-memory computing architectures (IMCs). An IMC noise model and associated SNR metrics are defined and their interrelationships analyzed to show that the accuracy of IMCs is fundamentally limited by the compute SNR (SNRa) of its analog core, and that activation, weight and output precision needs to be assigned appropriately for the final output SNR SNRT → SNRa. The minimum precision criterion (MPC) is proposed to minimize the ADC precision. Three in-memory compute models -charge summing (QS), current summing (IS) and charge redistribution (QR) -are shown to underlie most known IMCs. Noise, energy and delay expressions for the compute models are developed and employed to derive expressions for the SNR, ADC precision, energy, and latency of IMCs. The compute SNR expressions are validated via Monte Carlo simulations in a 65 nm CMOS process. For a 512 row SRAM array, it is shown that: 1) IMCs have an upper bound on their maximum achievable SNRa due to constraints on energy, area and voltage swing, and this upper bound reduces with technology scaling for QS-based architectures; 2) MPC enables SNRT → SNRa to be realized with minimal ADC precision; 3) QS-based (QR-based) architectures are preferred for low (high) compute SNR scenarios.

show abstract

Fundamental limits on the precision of in-memory architectures

Cited by 21 publications

References 29 publications

Benchmarking In-Memory Computing Architectures

Benchmarking In-Memory Computing Architectures

Adaptive Block Floating-Point for Analog Deep Learning Hardware

Fundamental Limits on Energy-Delay-Accuracy of In-memory Architectures in Inference Applications

Contact Info

Product

Resources

About