Bottom-Up and Top-Down Neural Processing Systems Design: Neuromorphic Intelligence as the Convergence of Natural and Artificial Intelligence

Abstract

While Moore's law has driven exponential computing power expectations, its nearing end calls for new avenues for improving the overall system performance. One of these avenues is the exploration of new alternative brain-inspired computing architectures that promise to achieve the flexibility and computational efficiency of biological neural processing systems. Within this context, neuromorphic intelligence represents a paradigm shift in computing based on the implementation of spiking neural network architectures tightly co-locating processing and memory. In this paper, we provide a comprehensive overview of the field, highlighting the different levels of granularity present in existing silicon implementations, comparing approaches that aim at replicating natural intelligence (bottom-up) versus those that aim at solving practical artificial intelligence applications (top-down), and assessing the benefits of the different circuit design styles used to achieve these goals. First, we present the analog, mixed-signal and digital circuit design styles, identifying the boundary between processing and memory through time multiplexing, in-memory computation and novel devices. Next, we highlight the key tradeoffs for each of the bottom-up and top-down approaches, survey their silicon implementations, and carry out detailed comparative analyses to extract design guidelines. Finally, we identify both necessary synergies and missing elements required to achieve a competitive advantage for neuromorphic edge computing over conventional machine-learning accelerators, and outline the key elements for a framework toward neuromorphic intelligence.

Online Training of Spiking Recurrent Neural Networks with Phase-Change Memory Synapses

“…On-chip training of spiking RNNs enable the low-power deployment of the intelligent computing systems at the edge with learning and adaptation capabilities [66,67]. In this work, we evaluated several widely used memristor update mechanisms on spiking RNNs based on the ideal gradient information calculated by the e-prop learning rule.…”

Review of spike-based neuromorphic computing for brain-inspired vision: biology, algorithms, and hardware

“…However, in spite of the recent rapid development of DNNs in the field of ML, specifically for computer vision tasks, their performance is not efficient enough compared with the biological brain in terms of power and speed. 18,31 While information in the biological brain is processed asynchronously and in parallel, information in DNNs is computed in a sequential form (i.e., computations at each layer of the network have to be completed on the input image before forwarding it to the following stage). Consequently, a significant delay appears in the network.…”

“…31 Consequently, a large body of research has been directed toward filling this gap, developing neuromorphic processors which are specifically customized for efficient SNN acceleration, moving from von Neumann architectures to distributed and compute-in-memory designs. 18 Neuromorphic processors can have digital, analog, or mixed signal implementations. The analog and mixed signal implementations replicate the biological brain better than digital designs in terms of power and speed.…”

“…Memristors exhibit behavioral similarity to chemical synapses, combining storage, adaptation, and physical connectivity in one device. 18 Table 3 shows a comparison of different two-terminal NVM technologies across several key metrics. Each device exploits different physics to achieve nonvolatile storage of information as a resistance state.…”

Matter & Mind Matter

“…In recent years, great progress has been made in the development of bio-inspired processors. Here, event-based spiking neural networks (SNNs) in the form of either mixed (analog and digital) or strictly digital signal processing provides novel opportunities for low-power data processing (Indiveri et al, 2011;Merolla et al, 2014;Pei et al, 2019;Kendall and Kumar, 2020;Frenkel et al, 2021).…”