Exploiting Non-idealities of Resistive Switching Memories for Efficient Machine Learning

Abstract

Novel computing architectures based on resistive switching memories (also known as memristors or RRAMs) have been shown to be promising approaches for tackling the energy inefficiency of deep learning and spiking neural networks. However, resistive switch technology is immature and suffers from numerous imperfections, which are often considered limitations on implementations of artificial neural networks. Nevertheless, a reasonable amount of variability can be harnessed to implement efficient probabilistic or approximate computing. This approach turns out to improve robustness, decrease overfitting and reduce energy consumption for specific applications, such as Bayesian and spiking neural networks. Thus, certain non-idealities could become opportunities if we adapt machine learning methods to the intrinsic characteristics of resistive switching memories. In this short review, we introduce some key considerations for circuit design and the most common non-idealities. We illustrate the possible benefits of stochasticity and compression with examples of well-established software methods. We then present an overview of recent neural network implementations that exploit the imperfections of resistive switching memory, and discuss the potential and limitations of these approaches.

A Review of Graphene‐Based Memristive Neuromorphic Devices and Circuits

“…[ 147 ] Although the stochasticity in DTD and CTC variations of fabricated memristors limits the ability to control the device's conductance (and other properties), it can be exploited in neuromorphic settings to overcome other issues, such as “overfitting.” Overfitting happens when a network fails to generalize because it is tuned mainly to the training data. [ 148 ] By utilizing this stochastic property, previous research has shown that overfitting problem can be overcome. [ 149 ]…”

“…Overfitting happens when a network fails to generalize because it is tuned mainly to the training data. [148] By utilizing this stochastic property, previous research has shown that overfitting problem can be overcome. [149] Some research efforts have been dedicated to eliminating variability by tuning the device fabrication process.…”

A High-Speed and High-Efficiency Diverse Error Margin Write-Verify Scheme for an RRAM-Based Neuromorphic Hardware Accelerator

“…The RRAM-based accelerator suffers from various sources of variations and noises [12]. It is difficult to precisely transfer learned weights into the hardware accelerator due to these variations.…”

Effect of memristor’s potentiation-depression curves peculiarities in the convergence of physical perceptrons

“…A critical issue related to the development of physical NN are the so-called 'non-idealities' of memristors, which can affect the NN performance [20,21,[27][28][29][30][31]. These include distinct P-D curves with constrained conductance windows ΔG and discrete number of conductance levels N (the granularity of the curves can be defined as the ratio between the latter and the former).…”

“…On the other hand, it is well established on standard NN that the addition of noise during the training can improve the NN generalization ability and reduce training losses [30]. A usual interpretation for the latter is that a reasonable amount of noise helps the system to avoid the Loss function to stabilize on local minima, favouring the convergence to a global minimum.…”