Non-Volatile Memory Array Based Quantization- and Noise-Resilient LSTM Neural Networks

Abstract

In cloud and edge computing models, it is important that compute devices at the edge be as power efficient as possible. Long short-term memory (LSTM) neural networks have been widely used for natural language processing, time series prediction and many other sequential data tasks. Thus, for these applications there is increasing need for low-power accelerators for LSTM model inference at the edge. In order to reduce power dissipation due to data transfers within inference devices, there has been significant interest in accelerating vector-matrix multiplication (VMM) operations using non-volatile memory (NVM) weight arrays. In NVM array-based hardware, reduced bit-widths also significantly increases the power efficiency. In this paper, we focus on the application of quantization-aware training algorithm to LSTM models, and the benefits these models bring in terms of resilience against both quantization error and analog device noise. We have shown that only 4-bit NVM weights and 4-bit ADC/DACs are needed to produce equivalent LSTM network performance as floating-point baseline. Reasonable levels of ADC quantization noise and weight noise can be naturally tolerated within our NVMbased quantized LSTM network. Benchmark analysis of our proposed LSTM accelerator for inference has shown at least 2.4× better computing efficiency and 40× higher area efficiency than traditional digital approaches (GPU, FPGA, and ASIC). Some other novel approaches based on NVM promise to deliver higher computing efficiency (up to ×4.7) but require larger arrays with potential higher error rates. Keywords-quantization, noise, LSTM, non-volatile memory, machine learning hardwareI.

A high throughput generative vector autoregression model for stochastic synapses

“…Furthermore, in digital systems current readouts are converted to finite resolution by analog to digital converters (ADCs). Due to constraints of power consumption and chip area, ADC resolution is often limited such that digitization is the dominant contributor to the total noise (Ma et al, 2019 ). Many additional noise sources can be considered, such as 1/ f noise (Wiefels et al, 2020 ), but at minimum the Johnson-Nyquist noise and the shot noise should be included because they represent a lower bound of noise amplitude impacting all systems.…”

A High Throughput Generative Vector Autoregression Model for Stochastic Synapses

“…Furthermore, in digital systems current readouts are converted to finite resolution by analog to digital converters (ADCs). Due to constraints of power consumption and chip area, ADC resolution is often limited such that digitization is the dominant contributor to the total noise [46]. Many additional noise sources can be considered, such as 1/f noise [47], but at minimum the Johnson-Nyquist noise and the shot noise should be included because they represent a lower bound of noise amplitude impacting all systems.…”

Quantization and sparsity-aware processing for energy-efficient NVM-based convolutional neural networks

“…Researchers have explored many NVM-based neural networks with limited precision designs in order to ease the hardware burden. Most studies considered low-to-mediumprecision quantization (usually 4 bits) (Jacob et al, 2018;Cai et al, 2019;Ma et al, 2019). Other studies on the other hand focused on the binary and ternary quantized conditions (Tang et al, 2017;Qin et al, 2020).…”