Towards spike-based machine intelligence with neuromorphic computing

Roy, Kaushik; Jaiswal, Akhilesh; Panda, Priyadarshini

doi:10.1038/s41586-019-1677-2

Cited by 1,023 publications

(653 citation statements)

References 127 publications

Supporting

Mentioning

649

Contrasting

Unclassified

Order By: Relevance

“…Carver Mead's pioneering efforts to emulate biological information processing using analog circuits (instead of logic gates used in digital computing) and leveraging the inherent device physics of Metal Oxide Semiconductor Field Effect Transistors (MOSFETs) established a new paradigm in computing hardware (Mead, 1990). Today, neuromorphic computing encompasses the use of novel nanotechnologies such as non-volatile memory devices and memristors (memory-resistors) that can mimic synapse-like memory and spiking temporal characteristics (Yang et al, 2013;Burr et al, 2017;Ziegler et al, 2018;Roy et al, 2019). Because of their unconventional "beyond von Neumann" architecture, which substantially reduces power requirements, such devices are also attractive for implementing Artificial Neural Network (ANN) algorithms, which require computationally-intensive training to learn input-output relationships, thereby mimicking neurons and synaptic connections in software (Xu et al, 2018).…”

Section: Neuromorphic Systems: Mimicking the Brain In Hardwarementioning

confidence: 99%

Topological Properties of Neuromorphic Nanowire Networks

et al. 2020

View full text Add to dashboard Cite

Graph theory has been extensively applied to the topological mapping of complex networks, ranging from social networks to biological systems. Graph theory has increasingly been applied to neuroscience as a method to explore the fundamental structural and functional properties of human neural networks. Here, we apply graph theory to a model of a novel neuromorphic system constructed from self-assembled nanowires, whose structure and function may mimic that of human neural networks. Simulations of neuromorphic nanowire networks allow us to directly examine their topology at the individual nanowire-node scale. This type of investigation is currently extremely difficult experimentally. We then apply network cartographic approaches to compare neuromorphic nanowire networks with: random networks (including an untrained artificial neural network); grid-like networks and the structural network of C. elegans. Our results demonstrate that neuromorphic nanowire networks exhibit a small-world architecture similar to the biological system of C. elegans, and significantly different from random and grid-like networks. Furthermore, neuromorphic nanowire networks appear more segregated and modular than random, grid-like and simple biological networks and more clustered than artificial neural networks. Given the inextricable link between structure and function in neural networks, these results may have important implications for mimicking cognitive functions in neuromorphic nanowire networks.

show abstract

Section: Neuromorphic Systems: Mimicking the Brain In Hardwarementioning

confidence: 99%

Topological Properties of Neuromorphic Nanowire Networks

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The growing interest for neuromorphic computing applications is driving a fundamental shift toward memory‐centric computer architectures, enabling efficient machine learning and analysis of large data sets. [ 1–3 ] Oxygen vacancy resistive random access memory (RRAM) is seen as a promising candidate for large capacity, nonvolatile memory technologies. [ 4–6 ] In this paper, we explore for the first time the possibility of cointegration of RRAM cells and metal‐oxide‐semiconductor field‐effect‐transistors (MOSFET) selectors onto III‐V vertical nanowires (VNWs) to benefit from the advantageous transistor properties.…”

Section: Figurementioning

confidence: 99%

Cross‐Point Arrays with Low‐Power ITO‐HfO₂ Resistive Memory Cells Integrated on Vertical III‐V Nanowires

Persson

Ram

Kilpi

et al. 2020

Adv Elect Materials

View full text Add to dashboard Cite

Vertical nanowires with cointegrated metal‐oxide‐semiconductor field‐effect‐transistor (MOSFET) selectors and nonvolatile resistive random access memory (RRAM) cells represent a promising candidate for fast, energy‐efficient, cross‐point memory cells. This paper explores indium‐tin‐oxide‐hafnium‐dioxide RRAM cells integrated onto arrays of indium‐arsenide (InAs) vertical nanowires with a resulting area of 0.06 µm2 per cell. For low current operation, an improved switching uniformity over the intrinsic self‐compliant behavior is demonstrated when using an external InAs nanowire MOSFET selector in series. The memory cells show consistent switching voltages below ±1 V and a switching cycle endurance of 106 is demonstrated. The developed fabrication scheme is fully compatible with low‐ON‐resistance vertical III‐V nanowire MOSFET selectors, where operational compatibility with the initial high‐field filament forming is established. Due to the small footprint of a vertical implementation, high density integration is achievable, and with a measured programming energy for 50 ns pulses at 0.49 pJ, the technology promises fast and ultralow power cross‐point memory arrays.

show abstract

“…8a). Experimental results suggest that most biological networks exhibit sparse spiking activity, a feature that is presumed to underlie their superior energy-efficiency [37][38][39][40][41][42]. We investigated whether surrogate gradients could instantiate SNNs in this biologically plausible, sparse activity regime.…”

Section: Optimal Sparse Spiking Activity Levels In Snnsmentioning

confidence: 99%

The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks

Zenke

Vogels

2020

Preprint

View full text Add to dashboard Cite

AbstractBrains process information in spiking neural networks. Their intricate connections shape the diverse functions these networks perform. In comparison, the functional capabilities of models of spiking networks are still rudimentary. This shortcoming is mainly due to the lack of insight and practical algorithms to construct the necessary connectivity. Any such algorithm typically attempts to build networks by iteratively reducing the error compared to a desired output. But assigning credit to hidden units in multi-layered spiking networks has remained challenging due to the non-differentiable nonlinearity of spikes. To avoid this issue, one can employ surrogate gradients to discover the required connectivity in spiking network models. However, the choice of a surrogate is not unique, raising the question of how its implementation influences the effectiveness of the method. Here, we use numerical simulations to systematically study how essential design parameters of surrogate gradients impact learning performance on a range of classification problems. We show that surrogate gradient learning is robust to different shapes of underlying surrogate derivatives, but the choice of the derivative’s scale can substantially affect learning performance. When we combine surrogate gradients with a suitable activity regularization technique, robust information processing can be achieved in spiking networks even at the sparse activity limit. Our study provides a systematic account of the remarkable robustness of surrogate gradient learning and serves as a practical guide to model functional spiking neural networks.

show abstract

Towards spike-based machine intelligence with neuromorphic computing

Cited by 1,023 publications

References 127 publications

Topological Properties of Neuromorphic Nanowire Networks

Topological Properties of Neuromorphic Nanowire Networks

Cross‐Point Arrays with Low‐Power ITO‐HfO₂ Resistive Memory Cells Integrated on Vertical III‐V Nanowires

The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks

Contact Info

Product

Resources

About

Towards spike-based machine intelligence with neuromorphic computing

Cited by 1,023 publications

References 127 publications

Topological Properties of Neuromorphic Nanowire Networks

Topological Properties of Neuromorphic Nanowire Networks

Cross‐Point Arrays with Low‐Power ITO‐HfO2 Resistive Memory Cells Integrated on Vertical III‐V Nanowires

The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks

Contact Info

Product

Resources

About

Cross‐Point Arrays with Low‐Power ITO‐HfO₂ Resistive Memory Cells Integrated on Vertical III‐V Nanowires