Issues of Implementing Neural Network Algorithms on Memristor Crossbars

Abstract

The property of natural parallelization of matrix−vector operations inherent in memristor crossbars creates opportunities for their effective use in neural network computing. Analog calculations are orders of magnitude faster in comparison to calculations on the central processor and on graphics accelerators. Besides, mathematical operations energy costs are significantly lower. The essential feature of analog computing is its low accuracy. In this regard, studying the dependence of neural network quality on the accuracy of setting its weights is relevant. The paper considers two convolutional neural networks trained on the MNIST (handwritten digits) and CIFAR_10 (airplanes, boats, cars, etc.) data sets. The first convolutional neural network consists of two convolutional layers, one subsample layer and two fully connected layers. The second one consists of four convolutional layers, two subsample layers and two fully connected layers. Calculations in convolutional and fully connected layers are performed through matrix−vector operations that are implemented on memristor crossbars. Sub−sampling layers imply the operation of finding the maximum value from several values. This operation can be implemented at the analog level. The process of training a neural network runs separately from data analysis. As a rule, gradient optimization methods are used at the training stage. It is advisable to perform calculations using these methods on CPU. When setting the weights, 3-4 precision bits are required to obtain an acceptable recognition quality in the case the network is trained on MNIST. 6−10 precision bits are required if the network is trained on CIFAR_10.

Interval Model of a Memristor Crossbar Network

“…A detailed description of the model is given in refs. [19–21,35]. In this article, the model has been modified to take into account the interval nature of the variables.…”

“…[ 16–18 ] Previously, the authors considered an approach to replacing the deterministic equation of state with a stochastic one by introducing a term responsible for additive (Gaussian) noise. [ 19–22 ] In this article, we propose to use the interval apparatus to take into account the scatter in the characteristics of elements. The idea is to introduce interval parameters into the mathematical model.…”

Mathematical Modeling of a Self-Learning Neuromorphic Network Based on Nanosized Memristive Elements with a 1T1R-Crossbar-Architecture

“…Ранее авторами уже была исследована возможность применения мемристоров для аналоговой реализации сверточных нейронных сетей второго поколения [5]. В работе [5] получены оценки характеристик мемристивных элементов, при которых их можно было бы использовать для аппаратной реализации соответствующих нейросетевых алгоритмов.…”