A 28-nm Convolutional Neuromorphic Processor Enabling Online Learning with Spike-Based Retinas

Frenkel, Charlotte; Legat, Jean-Didier; Bol, David

doi:10.1109/iscas45731.2020.9180440

Cited by 31 publications

(20 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This furthermore highlights that edge computing is an ideal use case for biologically-motivated algorithms, as an out-of-the-box application of feedback-alignment- and target-propagation-based algorithms currently does not scale to complex datasets (see Bartunov et al, 2018 for a recent review). We demonstrate this claim in Frenkel et al ( 2020 ) with the design of an event-driven convolutional processor that requires only 16.8–% power and 11.8–% silicon area overheads for on-chip online learning, a record-low overhead that is specifically enabled by DRTP, thus highlighting its low cost for edge computing devices. Finally, as DRTP can also be formulated as a three-factor learning rule for biologically-plausible learning, it is suitable for embedded neuromorphic computing, in which high-density synaptic plasticity can currently not be achieved without compromising learning performance (Frenkel et al, 2019b , c ).…”

Section: Introductionsupporting

confidence: 56%

“…Therefore, as opposed to increasing the resources of shallow-trained networks, DRTP offers a low-overhead training algorithm operating on small network topologies, ideally suiting edge-computing hardware requirements. These claims are proven in silico in Frenkel et al ( 2020 ), where implementing DRTP in an event-driven convolutional processor requires only 16.8–% power and 11.8–% silicon area overheads and allows demonstrating a favorable accuracy-power-area tradeoff compared to both on-chip online- and off-chip offline-trained conventional machine learning accelerators on the MNIST dataset.…”

Section: Discussionmentioning

confidence: 94%

See 1 more Smart Citation

Learning Without Feedback: Fixed Random Learning Signals Allow for Feedforward Training of Deep Neural Networks

2021

Self Cite

View full text Add to dashboard Cite

While the backpropagation of error algorithm enables deep neural network training, it implies (i) bidirectional synaptic weight transport and (ii) update locking until the forward and backward passes are completed. Not only do these constraints preclude biological plausibility, but they also hinder the development of low-cost adaptive smart sensors at the edge, as they severely constrain memory accesses and entail buffering overhead. In this work, we show that the one-hot-encoded labels provided in supervised classification problems, denoted as targets, can be viewed as a proxy for the error sign. Therefore, their fixed random projections enable a layerwise feedforward training of the hidden layers, thus solving the weight transport and update locking problems while relaxing the computational and memory requirements. Based on these observations, we propose the direct random target projection (DRTP) algorithm and demonstrate that it provides a tradeoff between accuracy and computational cost that is suitable for adaptive edge computing devices.

show abstract

Section: Introductionsupporting

confidence: 56%

Section: Discussionmentioning

confidence: 94%

Learning Without Feedback: Fixed Random Learning Signals Allow for Feedforward Training of Deep Neural Networks

2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…Most current neuromorphic systems are fully digital and typically allow one to simulate software-trained models without performance loss ( 38 , 39 ). This approach is flexible with regard to the SNN training schemes used ( 23 , 40 – 49 ), but to fully leverage recent advances in material sciences often requires dealing with analog or mixed-signal components ( 15 , 19 , 50 , 51 ).…”

Section: Discussionmentioning

confidence: 99%

Surrogate gradients for analog neuromorphic computing

Cramer

Billaudelle

Kanya

et al. 2022

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

To rapidly process temporal information at a low metabolic cost, biological neurons integrate inputs as an analog sum, but communicate with spikes, binary events in time. Analog neuromorphic hardware uses the same principles to emulate spiking neural networks with exceptional energy efficiency. However, instantiating high-performing spiking networks on such hardware remains a significant challenge due to device mismatch and the lack of efficient training algorithms. Surrogate gradient learning has emerged as a promising training strategy for spiking networks, but its applicability for analog neuromorphic systems has not been demonstrated. Here, we demonstrate surrogate gradient learning on the BrainScaleS-2 analog neuromorphic system using an in-the-loop approach. We show that learning self-corrects for device mismatch, resulting in competitive spiking network performance on both vision and speech benchmarks. Our networks display sparse spiking activity with, on average, less than one spike per hidden neuron and input, perform inference at rates of up to 85,000 frames per second, and consume less than 200 mW. In summary, our work sets several benchmarks for low-energy spiking network processing on analog neuromorphic hardware and paves the way for future on-chip learning algorithms.

show abstract

“…More efficient memory access is realized on an FPGA prototype in [ 27 ], by using a novel memory arbiter. A 28 nm SCNN processor is fabricated in [ 28 ], holding one CONV layer of 10 kernels for feature extraction, one pooling layer for dimension reduction, and two FC layers for feature classification. A systolic SCNN inference engine is proposed in [ 29 ], and two CONV layers with one FC layer are instantiated.…”

Section: Introductionmentioning

confidence: 99%

A Cost-Efficient High-Speed VLSI Architecture for Spiking Convolutional Neural Network Inference Using Time-Step Binary Spike Maps

Zhang

Yang

Shi

et al. 2021

Sensors

View full text Add to dashboard Cite

Neuromorphic hardware systems have been gaining ever-increasing focus in many embedded applications as they use a brain-inspired, energy-efficient spiking neural network (SNN) model that closely mimics the human cortex mechanism by communicating and processing sensory information via spatiotemporally sparse spikes. In this paper, we fully leverage the characteristics of spiking convolution neural network (SCNN), and propose a scalable, cost-efficient, and high-speed VLSI architecture to accelerate deep SCNN inference for real-time low-cost embedded scenarios. We leverage the snapshot of binary spike maps at each time-step, to decompose the SCNN operations into a series of regular and simple time-step CNN-like processing to reduce hardware resource consumption. Moreover, our hardware architecture achieves high throughput by employing a pixel stream processing mechanism and fine-grained data pipelines. Our Zynq-7045 FPGA prototype reached a high processing speed of 1250 frames/s and high recognition accuracies on the MNIST and Fashion-MNIST image datasets, demonstrating the plausibility of our SCNN hardware architecture for many embedded applications.

show abstract

A 28-nm Convolutional Neuromorphic Processor Enabling Online Learning with Spike-Based Retinas

Cited by 31 publications

References 25 publications

Learning Without Feedback: Fixed Random Learning Signals Allow for Feedforward Training of Deep Neural Networks

Learning Without Feedback: Fixed Random Learning Signals Allow for Feedforward Training of Deep Neural Networks

Surrogate gradients for analog neuromorphic computing

A Cost-Efficient High-Speed VLSI Architecture for Spiking Convolutional Neural Network Inference Using Time-Step Binary Spike Maps

Contact Info

Product

Resources

About