Binary‐Stochasticity‐Enabled Highly Efficient Neuromorphic Deep Learning Achieves Better‐than‐Software Accuracy

Abstract

In this work, the requirement of using high‐precision (HP) signals is lifted and the circuits for implementing deep learning algorithms in memristor‐based hardware are simplified. The use of HP signals is required by the backpropagation learning algorithm since the gradient descent learning rule relies on the chain product of partial derivatives. However, it is both challenging and biologically implausible to implement such an HP algorithm in noisy and analog memristor‐based hardware systems. Herein, it is demonstrated that the requirement for HP signals handling is not necessary and more efficient deep learning can be achieved when using a binary stochastic learning algorithm. The new algorithm proposed in this work modifies elementary neural network operations, which improves energy efficiency by two orders of magnitude compared to traditional memristor‐based hardware and three orders of magnitude compared to complementary metal–oxide–semiconductor‐based hardware. It also provides better accuracy in pattern recognition tasks than the HP learning algorithm benchmarks.

Difficulties and approaches in enabling learning-in-memory using crossbar arrays of memristors

“…It is further found that the low significant weights are not necessary to participate in the information forwarding and error backpropagation processes of the neural network [51,[62][63][64]. Thus, the gradient of the loss to the weight could be accumulated in a separate array outside of the crossbar of memristive devices, as shown in figure 6(f).…”