Is my Neural Network Neuromorphic? Taxonomy, Recent Trends and Future Directions in Neuromorphic Engineering

Bose, Sumon Kumar; Acharya, Jyotibdha; Basu, Arindam

doi:10.1109/ieeeconf44664.2019.9048891

Cited by 11 publications

(5 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Artificial Neural Networks (ANNs) rely on a large number of computations and data transfers, making their implementation in embedded systems, with stringent power, energy and area constraints, particularly challenging. Neuromorphic computing, such as bio-inspired algorithms and architectures, promise to improve the efficiency of hardware implementations of ANNs [1], [2]. In particular, analog implementations can achieve significant gains compared to fully-digital implementations [3].…”

Section: Introductionmentioning

confidence: 99%

Improving the Robustness of Neural Networks to Noisy Multi-Level Non-Volatile Memory-based Synapses

Dampfhoffer

Lopez

Mesquida

et al. 2023

2023 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

The implementation of Artificial Neural Networks (ANNs) using analog Non-Volatile Memories (NVMs) for synaptic weights storage promises improved energy-efficiency and higher density compared to fully-digital implementations. However, NVMs are prone to variability, resulting in a degradation of the accuracy of ANNs. In this paper, a general methodology to evaluate and enhance the accuracy of neural networks implemented with non-ideal multi-level NVMs is presented. A hardware fault model distinguishing two types of errors, namely static and dynamic, capturing the variability of NVMs is proposed. Considering various neural networks, it is shown that error-aware training highly increases the robustness to errors compared to a standard, error-agnostic, training. Moreover, Recurrent NNs (RNNs) and Spiking NNs (SNNs) are found to be inherently more robust to dynamic errors than Convolutional NNs (CNNs). In addition, new insights on the adaptability of neural networks to noisy multi-level NVMs are presented, which could further improve their robustness in this context. The methodology aims at providing tools for hardware-software co-design, paving the way for a broader use of multi-level NVM-based synapses.

show abstract

Section: Introductionmentioning

confidence: 99%

Improving the Robustness of Neural Networks to Noisy Multi-Level Non-Volatile Memory-based Synapses

Dampfhoffer

Lopez

Mesquida

et al. 2023

2023 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

show abstract

“…Neuromorphic computing systems are integrated circuits that implement the architecture of central nervous system in primates [14,22,65]. These systems facilitate energy-efficient computations using Spiking Neural Networks (SNN) [63] for power-constrained embedded devices.…”

Section: Introductionmentioning

confidence: 99%

Design-Technology Co-Optimization for NVM-based Neuromorphic Processing Elements

Song¹,

Balaji²,

Das³

et al. 2022

Preprint

View full text Add to dashboard Cite

An emerging use-case of machine learning (ML) is to train a model on a high-performance system and deploy the trained model on energy-constrained embedded systems. Neuromorphic hardware platforms, which operate on principles of the biological brain, can significantly lower the energy overhead of a machine learning inference task, making these platforms an attractive solution for embedded ML systems. We present a designtechnology tradeoff analysis to implement such inference tasks on the processing elements (PEs) of a Non-Volatile Memory (NVM)-based neuromorphic hardware. Through detailed circuit-level simulations at scaled process technology nodes, we show the negative impact of technology scaling on the information-processing latency, which impacts the quality-of-service (QoS) of an embedded ML system. At a finer granularity, the latency inside a PE depends on 1) the delay introduced by parasitic components on its current paths, and 2) the varying delay to sense different resistance states of its NVM cells. Based on these two observations, we make the following three contributions. First, on the technology front, we propose an optimization scheme where the NVM resistance state that takes the longest time to sense is set on current paths having the least delay, and vice versa, reducing the average PE latency, which improves the QoS. Second, on the architecture front, we introduce isolation transistors within each PE to partition it into regions that can be individually power-gated, reducing both latency and energy. Finally, on the system-software front, we propose a mechanism to leverage the proposed technological and architectural enhancements when implementing a machine-learning inference task on neuromorphic PEs of the hardware. Evaluations with a recent neuromorphic hardware architecture show that our proposed design-technology co-optimization approach improves both performance and energy efficiency of machine-learning inference tasks without incurring high cost-per-bit. CCS Concepts: • Hardware → Neural systems; Emerging languages and compilers; Emerging tools and methodologies; • Computer systems organization → Data flow architectures; Neural networks.

show abstract

“…This encourages competition among companies, such as Intel, IBM, and others to propose new hardware alternatives, leading to the emergence of commercially available deep learning accelerators (Barry et al, 2015;Jouppi et al, 2017) and neuromorphic chips (Esser et al, 2016;Davies et al, 2018;Pei et al, 2019). Deep learning accelerators are application specific integrated circuits (ASICs) tailored for artificial neural networks (ANN), whereas, neuromorphic chips can fall in two categories (Bose et al, 2019): (1) ASICs with biologically inspired spiking neural networks (SNN), which contain networks of neurons and synapses for computation and communication, or (2) ASICs with analog computing by exploiting dense nonvolatile memory based crossbars to accelerate matrix-vector multiplications. Our paper is not concerned with any specific hardware but any neuromorphic architecture relying on analog crossbars for matrix-vector multiplications.…”

Section: Introductionmentioning

confidence: 99%

HFNet: A CNN Architecture Co-designed for Neuromorphic Hardware With a Crossbar Array of Synapses

et al. 2020

Self Cite

View full text Add to dashboard Cite

The hardware-software co-optimization of neural network architectures is a field of research that emerged with the advent of commercial neuromorphic chips, such as the IBM TrueNorth and Intel Loihi. Development of simulation and automated mapping software tools in tandem with the design of neuromorphic hardware, whilst taking into consideration the hardware constraints, will play an increasingly significant role in deployment of system-level applications. This paper illustrates the importance and benefits of co-design of convolutional neural networks (CNN) that are to be mapped onto neuromorphic hardware with a crossbar array of synapses. Toward this end, we first study which convolution techniques are more hardware friendly and propose different mapping techniques for different convolutions. We show that, for a seven-layered CNN, our proposed mapping technique can reduce the number of cores used by 4.9-13.8 times for crossbar sizes ranging from 128 × 256 to 1,024 × 1,024, and this can be compared to the toeplitz method of mapping. We next develop an iterative co-design process for the systematic design of more hardware-friendly CNNs whilst considering hardware constraints, such as core sizes. A python wrapper, developed for the mapping process, is also useful for validating hardware design and studies on traffic volume and energy consumption. Finally, a new neural network dubbed HFNet is proposed using the above co-design process; it achieves a classification accuracy of 71.3% on the IMAGENET dataset (comparable to the VGG-16) but uses 11 times less cores for neuromorphic hardware with core size of 1,024 × 1,024. We also modified the HFNet to fit onto different core sizes and report on the corresponding classification accuracies. Various aspects of the paper are patent pending.

show abstract

Is my Neural Network Neuromorphic? Taxonomy, Recent Trends and Future Directions in Neuromorphic Engineering

Cited by 11 publications

References 31 publications

Improving the Robustness of Neural Networks to Noisy Multi-Level Non-Volatile Memory-based Synapses

Improving the Robustness of Neural Networks to Noisy Multi-Level Non-Volatile Memory-based Synapses

Design-Technology Co-Optimization for NVM-based Neuromorphic Processing Elements

HFNet: A CNN Architecture Co-designed for Neuromorphic Hardware With a Crossbar Array of Synapses

Contact Info

Product

Resources

About