In-situ Stochastic Training of MTJ Crossbar based Neural Networks

Abstract

Owing to high device density, scalability and non-volatility, Magnetic Tunnel Junction-based crossbars have garnered signi cant interest for implementing the weights of an arti cial neural network. e existence of only two stable states in MTJs implies a high overhead of obtaining optimal binary weights in so ware. We illustrate that the inherent parallelism in the crossbar structure makes it highly appropriate for in-situ training, wherein the network is taught directly on the hardware. It leads to signi cantly smaller training overhead as the training time is independent of the size of the network, while also circumventing the e ects of alternate current paths in the crossbar and accounting for manufacturing variations in the device. We show how the stochastic switching characteristics of MTJs can be leveraged to perform probabilistic weight updates using the gradient descent algorithm. We describe how the update operations can be performed on crossbars both with and without access transistors and perform simulations on them to demonstrate the e ectiveness of our techniques. e results reveal that stochastically trained MTJ-crossbar NNs achieve a classi cation accuracy nearly same as that of realvalued-weight networks trained in so ware and exhibit immunity to device variations.

Training of quantized deep neural networks using a magnetic tunnel junction-based synapse

“…Previous works used the stochastic behavior of the STT-MTJ, or other memristive technologies such as resistive RAM (RRAM), to implement hardware accelerators for BNNs [9][10][11][12]. In [9], the research focus was on the architecture level of BNN accelerators, without supporting training.…”

“…A recently proposed MTJ-based binary synapse comprising a single transistor and a single MTJ device (1T1R) [12] supports training QNNs with binary weights and real value activations. Mondal and Srivastava [12] exploited analog computation to support processing near memory (PNM). Their design, however, requires two update operations to execute the SGD updates.…”

Methodology for Realizing VMM with Binary RRAM Arrays: Experimental Demonstration of Binarized-ADALINE Using OxRAM Crossbar

“…In particular, emerging resistive memory (RRAM) nanodevice based arrays offer an efficient and compact option for performing VMM operations in hardware. Several research groups [2]- [6] have demonstrated this analog computing method for a variety of applications such as linear equation solver [6], image processing [7], data compression [8], feature extraction [9], neural network inference [10], in-situ training [10], [11], etc. Multiple emerging memory nanodevice technologies have been utilized for this application : OxRAM [1], MRAM [11], PCM [12], Ferroelectric FET [13], ECRAM [14], Flash [15], etc.…”

“…Several research groups [2]- [6] have demonstrated this analog computing method for a variety of applications such as linear equation solver [6], image processing [7], data compression [8], feature extraction [9], neural network inference [10], in-situ training [10], [11], etc. Multiple emerging memory nanodevice technologies have been utilized for this application : OxRAM [1], MRAM [11], PCM [12], Ferroelectric FET [13], ECRAM [14], Flash [15], etc. Owing to their higher device density, crossbar structures are preferred for VMM applications [16].…”

A Novel Compound Synapse Using Probabilistic Spin–Orbit-Torque Switching for MTJ-Based Deep Neural Networks

“…In particular, we have explored the accuracy of a 784×200×10 DBN as shown in figure 1 for MNIST pattern recognition, noting that the realization of a synaptic network with MRAM elements presents an attractive opportunity to build an all-MRAM based DBN. It is also worthwhile noticing that the compound synaptic structure proposed here can readily be combined with other neuromorphic computing architectures as discussed in [10] [12][13] [14].…”