Surrogate gradients for analog neuromorphic computing

Abstract

To rapidly process temporal information at a low metabolic cost, biological neurons integrate inputs as an analog sum, but communicate with spikes, binary events in time. Analog neuromorphic hardware uses the same principles to emulate spiking neural networks with exceptional energy efficiency. However, instantiating high-performing spiking networks on such hardware remains a significant challenge due to device mismatch and the lack of efficient training algorithms. Surrogate gradient learning has emerged as a promising training strategy for spiking networks, but its applicability for analog neuromorphic systems has not been demonstrated. Here, we demonstrate surrogate gradient learning on the BrainScaleS-2 analog neuromorphic system using an in-the-loop approach. We show that learning self-corrects for device mismatch, resulting in competitive spiking network performance on both vision and speech benchmarks. Our networks display sparse spiking activity with, on average, less than one spike per hidden neuron and input, perform inference at rates of up to 85,000 frames per second, and consume less than 200 mW. In summary, our work sets several benchmarks for low-energy spiking network processing on analog neuromorphic hardware and paves the way for future on-chip learning algorithms.

The BrainScaleS-2 Accelerated Neuromorphic System With Hybrid Plasticity

“…Subsequent work applied this training procedure to other neuron models or used other pseudoderivatives (Bellec et al, 2018;Neftci et al, 2019). The in-the-loop training approach was realized on the BrainScaleS platform by Schmitt et al (2017) for rate-based models and (Cramer et al, 2022) for SNNs employing individual spikes for information transfer and processing, as summarized in Section 3.3.1. Petrovici et al (2016) related stochastically stimulated and recurrently connected populations of LIF neurons in the highconductance state to Restricted Boltzmann Machines.…”

“…• Time-to-first spike (Göltz et al, 2021) • Surrogate-Gradient-Based Learning (Cramer et al, 2022) • Analog ANN training (Weis et al, 2020) They differ in which measurements are necessary and what model of the physical system is used. In the time-to-first spike gradient-based training scheme, which we won't discuss in detail here, the essential idea is that it is possible to compute the derivative of the spike time with respect to input weights based on an analytical expression of the spike time.…”

“…We benchmarked our learning framework on several challenging datasets. For example, we trained feed-forward SNN with a hidden layer composed of 246 LIF neurons on the handwritten MNIST digits (LeCun et al 1998 , Figure 8A ), where we reached a test accuracy of (97.6 ± 0.1)% (Cramer et al, 2022 ). Notably, we observed a time-to-decision of less than 7 μs ( Figure 8B ).…”

“…Moreover, our framework could be extended to facilitate the training of recurrent SNNs. Specifically, we trained a recurrent SNN with a single hidden layer composed of 186 LIF neurons on the SHD dataset (Cramer et al, 2020b ), where we reached a test performance of (80.0 ± 1.0)% (Cramer et al, 2022 ).…”

Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision

“…1 for an illustration). This is made possible by combining standard backpropagation with recently developed surrogate gradient methods to train deep SNNs [19,20,21,22,23,24]. The advantage of the hybrid approach is that early layers can operate in the extremely efficient event-based computing paradigm, which can run on special purpose hardware implementations [25,26,27,28,29,30].…”