Comprehending In-memory Computing Trends via Proper Benchmarking

Shanbhag, Naresh R.; Roy, Saion K.

doi:10.1109/cicc53496.2022.9772817

Cited by 16 publications

(7 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…10. Confidence interval and confidence level vs. number of test samples (from [95]). downsample the dataset.…”

Section: Network Accuracy (P D )mentioning

confidence: 99%

“…In this paper, we propose a rigorous benchmarking methodology for IMCs comprising: 1) a compositional view of IMCs that enables one to parse an IMC design into its canonical components; 2) a set of benchmarking metrics to quantify the performance, efficiency, and accuracy of IMCs; and 3) a strategy for analyzing the reported IMC data and metrics. We apply the proposed benchmarking methodology on an extensive database of metrics collected from > 70 machine learning IC designs published in major circuits conferences since 2018 in order to comprehend trends and identify challenges in This paper builds upon a preliminary version presented in [95] by incorporating 2022 data (data until 2021 was analyzed in [95]) and providing rationale to explain various observed trends. Furthermore, the database and Python code to generate the plots in this paper is also being made available at: [a link to GitHub repository will be provided here].…”

Section: Introductionmentioning

confidence: 99%

“…IMC design. First, trends in SRAM-based IMCs [14]-[53] are drawn, followed by comparisons between eNVM-based IMCs [54]-[71], eDRAM-based IMCs [72]-[74], and with recent digital accelerators [75]-[94].This paper builds upon a preliminary version presented in[95] by incorporating 2022 data (data until 2021 was analyzed in[95]) and providing rationale to explain various observed trends. Furthermore, the database and Python code to generate the plots in this paper is also being made available at: [a link to GitHub repository will be provided here].The outline of this paper is as follows: Section II describes a compositional view of IMCs and associated metrics.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Benchmarking In-Memory Computing Architectures

Shanbhag

Roy

2022

IEEE Open J. Solid-State Circuits Soc.

Self Cite

View full text Add to dashboard Cite

In-memory computing (IMC) architectures have emerged as a compelling platform to implement energy efficient machine learning (ML) systems. However, today, the energy efficiency gains provided by IMC designs seem to be leveling off and it is not clear what the limiting factors are. The conceptual complexity of IMCs combined with the absence of a rigorous benchmarking methodology makes it difficult to gauge progress and identify bottlenecks in this exciting field. This paper presents a benchmarking methodology for IMCs comprising: 1) a compositional view of IMCs that enables one to parse an IMC design into its canonical components; 2) a set of benchmarking metrics to quantify the performance, efficiency, and accuracy of IMCs; and 3) a strategy for analyzing the reported IMC data and metrics. We apply the proposed benchmarking methodology on an extensive database of IMC metrics extracted from > 70 IC designs published since 2018, in order to infer and comprehend trends in this area.Our benchmarking effort indicates: 1) SRAM-based IMCs show a clear win in terms of energy efficiency and compute density over digital accelerators at the bank-level but the energy efficiency gap reduces dramatically when comparing at the processor level; 2) eNVM-based IMCs lag behind SRAM-based IMCs in terms of both energy efficiency and compute density, and surprisingly lag digital accelerators in terms of compute density; 3) the compute (bank-level) accuracy of IMCs, though a critical metric, is pervasively neglected in publications as is the energy vs. accuracy trade-off inherent to IMCs.

show abstract

“…10. Confidence interval and confidence level vs. number of test samples (from [95]). downsample the dataset.…”

Section: Network Accuracy (P D )mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Benchmarking In-Memory Computing Architectures

Shanbhag

Roy

2022

IEEE Open J. Solid-State Circuits Soc.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The information content in a MAC, as noted in [47], can be given(with an exception for when B X = 1) by:-…”

Section: Problem Setupmentioning

confidence: 99%

“…A comprehensive review of trends in recent IMC works can be found in [47], which notes that most IMC works implement B ADC bit ADCs to quantise MACs, with B ADC ≪ B I(col) . However, as IMC works strive to maximise row parallelism to amortise costs, B I(col) will increase.…”

Section: Problem Setupmentioning

confidence: 99%

Input Conditioned Subranging and Skewed Quantisation of MACs in IMC

Sundar¹,

Viraraghavan²,

Vijayakumar³

2022

Preprint

View full text Add to dashboard Cite

<p>In-Memory Computation(IMC) of Neural-Network(NN) inference is done by performing the Multiply-ACcumulate(MAC) operation in the analog domain. Parallelly digitising MAC voltages by fitting Analog to Digital Converters(ADCs) within dense memory-pitches is a fundamental challenge for IMC engines. IMC works thus far rely on clipping the MAC-PDF to reduce the dynamic range, reducing the per data-line(DL) ADC precision requirement. In this work, we show that the per-DL ADC precision can be reduced even further by focusing on quantising the input Conditioned MAC-PDF(CMPDF), which spans a sub-range in the total MAC-PDF. We demonstrate on hardware a technique to locate the CMPDF in one-shot by tracking its mean. We show that quantisation levels about the CMPDF mean can be skewed to only span the portion of CMPDF that yields positive ReLU inputs, provided MACs are implemented as complete sums. Compared to symmetrically spanning CMPDF, this requires 20% to 40% fewer references at iso-accuracy for the investigated neural network layers. Hardware measured results for Fully-Connected NN inference on MNIST yielded < 1% accuracy drop when MAC-voltages were quantised with 4 bit references about the CMPDF mean as compared to full-range 6 bit ADC sensing the MAC-voltages. MATLAB evaluation using 3.5 to 4.2 bit CMPDF quantisation for MNIST-FCNN, CIFAR-10 Resnet-20 and VGG-11 inference yielded < 1% accuracy drop as compared to full-range 6 to 7 bit MAC quantisation. </p>

show abstract

A real-time and energy-efficient SRAM with mixed-signal in-memory computing near CMOS sensors

Diaz-Madrid,

Domenech-Asensi,

Ruiz-Merino

et al. 2024

J Real-Time Image Proc

View full text Add to dashboard Cite

In-memory computing (IMC) represents a promising approach to reducing latency and enhancing the energy efficiency of operations required for calculating convolution products of images. This study proposes a fully differential current-mode architecture for computing image convolutions across all four quadrants, intended for deep learning applications within CMOS imagers utilizing IMC near the CMOS sensor. This architecture processes analog signals provided by a CMOS sensor without the need for analog-to-digital conversion. Furthermore, it eliminates the necessity for data transfer between memory and analog operators as convolutions are computed within modified SRAM memory. The paper suggests modifying the structure of a CMOS SRAM cell by incorporating transistors capable of performing multiplications between binary (−1 or +1) weights and analog signals. Modified SRAM cells can be interconnected to sum the multiplication results obtained from individual cells. This approach facilitates connecting current inputs to different SRAM cells, offering highly scalable and parallelized calculations. For this study, a configurable module comprising nine modified SRAM cells with peripheral circuitry has been designed to calculate the convolution product on each pixel of an image using a $$3 \times 3$$ 3 × 3 mask with binary values (−1 or 1). Subsequently, an IMC module has been designed to perform 16 convolution operations in parallel, with input currents shared among the 16 modules. This configuration enables the computation of 16 convolutions simultaneously, processing a column per cycle. A digital control circuit manages both the readout or memorization of digital weights, as well as the multiply and add operations in real-time. The architecture underwent testing by performing convolutions between binary masks of 3 × 3 values and images of 32 × 32 pixels to assess accuracy and scalability when two IMC modules are vertically integrated. Convolution weights are stored locally as 1-bit digital values. The circuit was synthesized in 180 nm CMOS technology, and simulation results indicate its capability to perform a complete convolution in 3.2 ms, achieving an efficiency of 11,522 1-b TOPS/W (1-b tera-operations per second per watt) with a similarity to ideal processing of 96%.

show abstract

Comprehending In-memory Computing Trends via Proper Benchmarking

Cited by 16 publications

References 58 publications

Benchmarking In-Memory Computing Architectures

Benchmarking In-Memory Computing Architectures

Input Conditioned Subranging and Skewed Quantisation of MACs in IMC

A real-time and energy-efficient SRAM with mixed-signal in-memory computing near CMOS sensors

Contact Info

Product

Resources

About