Low-Power, Adaptive Neuromorphic Systems: Recent Progress and Future Directions

Abstract

In this paper, we present a survey of recent works in developing neuromorphic or neuro-inspired hardware systems.In particular, we focus on those systems which can either learn from data in an unsupervised or online supervised manner. We present algorithms and architectures developed specially to support on-chip learning. Emphasis is placed on hardware friendly modifications of standard algorithms, such as backpropagation, as well as novel algorithms, such as structural plasticity, developed specially for low-resolution synapses. We cover works related to both spike-based and more traditional non-spike-based algorithms. This is followed by developments in novel devices, such as floating-gate MOS, memristors, and spintronic devices. CMOS circuit innovations for on-chip learning and CMOS interface circuits for post-CMOS devices, such as memristors, are presented. Common architectures, such as crossbar or island style arrays, are discussed, along with their relative merits and demerits. Finally, we present some possible applications of neuromorphic hardware, such as brain-machine interfaces, robotics, etc., and identify future research trends in the field.

Spiking Neural Networks Hardware Implementations and Challenges

“…Several models describe the neuron functions, with a varying degree of complexity [52,53,75]. A large amount of them have already been transposed onto hardware [11,20,31,73,198]. The Leaky Integrate and Fire (LIF) model has encountered a special interest among hardware designers [1,15,38,122], we thus discuss its working principle below.…”

HFNet: A CNN Architecture Co-designed for Neuromorphic Hardware With a Crossbar Array of Synapses

“…Over the past decade, GPUs have emerged as a major hardware resource for deep learning tasks. However, fields, such as the internet of things (IoT) and edge computing are constantly in need of more efficient neural-network-specific hardware (Basu et al, 2018;Deng et al, 2018;Alyamkin et al, 2019;Roy et al, 2019). This encourages competition among companies, such as Intel, IBM, and others to propose new hardware alternatives, leading to the emergence of commercially available deep learning accelerators (Barry et al, 2015;Jouppi et al, 2017) and neuromorphic chips (Esser et al, 2016;Davies et al, 2018;Pei et al, 2019).…”

Tunneling-based CMOS Floating Gate Synapse for Low Power Spike Timing Dependent Plasticity

“…The latter is a famous method used to modify the strength of synapses depending on the relative times of spiking of the involved neurons [18]. The memory element is a floating gate that stores quasi-permanently a charge and it is one of the main candidates for neuromorphic circuits [19]- [21] thanks to the full compatibility with the current CMOS technology. Differently from previously reported floating gate synapses [14], [20]- [22], the stored charge is modified through Fowler-Nordheim tunneling effect [23] avoiding the large current required by hot-carrier injection [24].…”