Towards Efficient RRAM-based Quantized Neural Networks Hardware: State-of-the-art and Open Issues

Abstract

The increasing amount of data processed on edge and demand for reducing the energy consumption for large neural network architectures have initiated the transition from traditional von Neumann architectures towards in-memory computing paradigms. Quantization is one of the methods to reduce power and computation requirements for neural networks by limiting bit precision. Resistive Random Access Memory (RRAM) devices are great candidates for Quantized Neural Networks (QNN) implementations. As the number of possible conductive states in RRAMs is limited, a certain level of quantization is always considered when designing RRAM-based neural networks. In this work, we provide a comprehensive analysis of state-of-the-art RRAMbased QNN implementations, showing where RRAMs stand in terms of satisfying the criteria of efficient QNN hardware. We cover hardware and device challenges related to QNNs and show the main unsolved issues and possible future research directions.

Towards Efficient In-Memory Computing Hardware for Quantized Neural Networks: State-of-the-Art, Open Challenges and Perspectives

“…These non-idealities include endurance issues, conductance drift, problems of stacking at fault values, and non-linearity of the switching curve. However, nonvolatile memories allow multi-level storage, scalability, and high computational density [8]. For example, if the cell size of SRAM is approximately 124F 2 , where F is a technology feature size, RRAM cell with the selector device (1T1R cell) has a size of 12F 2 [9].…”

“…A. Binarized Neural Networks and QNNs with Ternary Weights Binarized neural network (BNN) is a type of QNN relying on 1-bit weights (+1 and -1 in software) and activations [8]. The fabricated binary SRAM-based IMC designs are shown in [31], [32].…”

“…The other RRAM-based fabricated prototype of BNN with flash ADCs is shown in [35]. The application of flash ADCs is acceptable for BNNs, however for higher-bit designs, flash ADCs can bring significant area and power overhead [8]. A fabricated MRAM-based IMC macro for 1-bit operations is shown in [36].…”

“…The RRAM-based IMC accelerator level designs with several levels of architecture hierarchy are shown in [50], [51], [28], [61]. These QNN designs include the datapath, routing, and arrangement of the network nodes are considered in addition to controlling and computation peripherals [8].…”

“…In crossbar macros, the hardware overhead comes from ADC and partial sums, when high-precision weights split into lowprecision crossbar cells [9]. ADC design improvements, implementation of low-power converters, reducing the number of ADCs and ADC resolution, and relying on approximate computing can benefit IMC hardware designs [8]. Fig.…”