Training of quantized deep neural networks using a magnetic tunnel junction-based synapse

Abstract

Quantized neural networks (QNNs) are being actively researched as a solution for the computational complexity and memory intensity of deep neural networks. This has sparked efforts to develop algorithms that support both inference and training with quantized weight and activation values, without sacrificing accuracy. A recent example is the GXNOR framework for stochastic training of ternary and binary neural networks (TNNs and BNNs, respectively). In this paper, we show how magnetic tunnel junction (MTJ) devices can be used to support QNN training. We introduce a novel hardware synapse circuit that uses the MTJ stochastic behaviour to support the quantize update. The proposed circuit enables processing near memory (PNM) of QNN training, which subsequently reduces data movement. We simulated MTJ-based stochastic training of a TNN over the MNIST, SVHN, and CIFAR10 datasets and achieved an accuracy of 98.61 % , 93.99 % and 83.02 % , respectively (less than 1 % degradation compared to the GXNOR algorithm). We evaluated the synapse array performance potential and showed that the proposed synapse circuit can train TNNs in situ, with 18.3 T O P s W for feedforward and 3 T O P s W for weight update.

Quantum materials for energy-efficient neuromorphic computing

“…109 Magnetic tunnel junctions emulate binary weights by switching between two magnetization states. [110][111][112] The intrinsic stochastic nature of magnetization switching in twostate magnetic tunnel junctions can also be leveraged for learning. 113 Memristive behavior is obtained by modifying the magnetization texture to obtain gradual switching via spin-torque 114,115 or spin-orbit torques.…”