Machine learning using magnetic stochastic synapses

Abstract

The impressive performance of artificial neural networks has come at the cost of high energy usage and CO2 emissions. Unconventional computing architectures, with magnetic systems as a candidate, have potential as alternative energy-efficient hardware, but, still face challenges, such as stochastic behaviour, in implementation. Here, we present a methodology for exploiting the traditionally detrimental stochastic effects in magnetic domain-wall motion in nanowires. We demonstrate functional binary stochastic synapses alongside a gradient learning rule that allows their training with applicability to a range of stochastic systems. The rule, utilising the mean and variance of the neuronal output distribution, finds a trade-off between synaptic stochasticity and energy efficiency depending on the number of measurements of each synapse. For single measurements, the rule results in binary synapses with minimal stochasticity, sacrificing potential performance for robustness. For multiple measurements, synaptic distributions are broad, approximating better-performing continuous synapses. This observation allows us to choose design principles depending on the desired performance and the device's operational speed and energy cost. We verify performance on physical hardware, showing it is comparable to a standard neural network.

Editorial: Focus on Neuromorphic Circuits and Systems using Emerging Devices

“…A quantitative electrical model explains this memory-based phenomenon, offering insights into material mimesis of neural communications and learning-memory functions in the human brain. [8]: Ellis et al introduce a methodology for harnessing stochastic effects in magnetic domain-wall motion in nanowires to create energy-efficient neural network hardware. It presents binary stochastic synapses with a gradient learning rule based on neuronal output distribution statistics.…”

Progress in Spin Logic Devices Based on Domain-Wall Motion

“…More information can be found in the review by Venkat et al [ 33 ]. DW devices can indeed be used to implement neuromorphic computing (NC), mainly by providing synaptic behavior with multilevel resistance [ 96 , 97 , 98 , 99 , 100 , 101 , 102 ] or by providing a sigmoid passing probability [ 103 , 104 ]. DW devices also have applications for stochastic computing (SC) [ 105 , 106 ], as well as reservoir computing (RC) [ 107 , 108 ].…”

Magnetic domain walls: types, processes and applications

“…The stochastic dynamics of DWs at notches in IM nanowires used have also been used to demonstrate functional binary stochastic synapses by Ellis et al [237] (Type:Expt). They used magnetic field driven nucleation and Oersted fields of current pulses for DW dynamics in an IM permalloy nanowire with an artificial notch to realise the sigmoid-like passing probability of a DW using MOKE.…”