Operation Scheme of Multi-Layer Neural Networks Using NAND Flash Memory as High-Density Synaptic Devices

Abstract

We propose a designing of multi-layer neural networks using 2D NAND flash memory cell as a high-density and reliable synaptic device. Our operation scheme eliminates the waste of NAND flash cells and allows analogue input values. A 3-layer perceptron network with 40,545 synapses is trained on a MNIST database set using an adaptive weight update method for hardware-based multi-layer neural networks. The conductance response of NAND flash cells is measured and it is shown that the unidirectional conductance response is suitable for implementing multi-layer neural networks using NAND flash memory cells as synaptic devices. Using an online-learning, we obtained higher learning accuracy with NAND synaptic devices compared to that with a memristor-based synapse regardless of weight update methods. Using an adaptive weight update method based on a unidirectional conductance response, we obtained a 94.19% learning accuracy with NAND synaptic devices. This accuracy is comparable to 94.69% obtained by synapses based on the ideal perfect linear device. Therefore, NAND flash memory which is mature technology and has great advantage in cell density can be a promising synaptic device for implementing high-density multi-layer neural networks. INDEX TERMS Neuromorphic, NAND flash memory, deep neural networks (DNNs), synaptic device, deep learning, multi-layer neural networks, hardware-based neural network.

3D AND-Type Stacked Array for Neuromorphic Systems

“…Read disturb as well as program disturb can change the conductance of a synaptic device, reducing its accuracy. When implementing a synapse array with a NAND-type array, a pass voltage must be applied to de-selected cells of the same string during the inference operation, causing a read disturb ( Figure 10 a) [ 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 ]. However, in the proposed structure, there is little risk of a read disturb because there is no need to apply pass voltage to the word lines of de-selected cells ( Figure 10 b).…”

“… ( a ) Schematic of neural networks using NAND flash memory [ 40 ]; ( b ) Schematic of the proposed structure: pass voltage is not used on the de-selected cells and read disturb error can be suppressed. …”

NAND Flash Based Novel Synaptic Architecture for Highly Robust and High-Density Quantized Neural Networks With Binary Neuron Activation of (1, 0)

“…5. The estimated device variation (σw/μw) of the W2 and W3 levels are 3.04% and 1.88% respectively, based on the measured data with the assumption of a Gaussian distribution [24], [25]. Fig.…”

“…However, in the weighted sum equation of the DNN model, the weighted sum output is linearly proportional to the input value. Therefore, the amplitude of the input in the DNN model cannot be encoded as an analogue amplitude of the input voltage in the neuromorphic system [24]. We adopt a binary activation of (1, 0) which can be applied to the nonlinear I-V curve by assuming that the input of 1 and 0 correspond to the turn-on (Von) and turn-off voltage (Voff) of the SSL device, respectively.…”

Neuromorphic Computing Using NAND Flash Memory Architecture With Pulse Width Modulation Scheme

“…The estimated device variation (σ w /µ w ) of W2 is 3.43%, and W3 is 1.68% based on the statistical parameters extracted from the measurement data. In this estimation, we assume that the conductance distribution of NAND cells follows a Gaussian distribution (Lee et al, 2019b). Figure 6 represents a PWM circuit consisting of a sawtooth generator, a differential amplifier, and a level shifter.…”