Keys to accurate feature extraction using residual spiking neural networks

Vicente-Sola, Alex; Manna, Davide L.; Kirkland, Paul; Caterina, Gaetano Di; Bihl, Trevor J.

doi:10.1088/2634-4386/ac8bef

Cited by 12 publications

(4 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A typical example is that of 34 , where the researchers use a 38-layer ResNet. The scheme uses batch normalization through time (BNTT), instead of simple batchnorm to improve performance.…”

Section: Related Work and Problem Definitionmentioning

confidence: 99%

A low cost neuromorphic learning engine based on a high performance supervised SNN learning algorithm

Siddique

Vai

Pun

2023

Sci Rep

View full text Add to dashboard Cite

Spiking neural networks (SNNs) are more energy- and resource-efficient than artificial neural networks (ANNs). However, supervised SNN learning is a challenging task due to non-differentiability of spikes and computation of complex terms. Moreover, the design of SNN learning engines is not an easy task due to limited hardware resources and tight energy constraints. In this article, a novel hardware-efficient SNN back-propagation scheme that offers fast convergence is proposed. The learning scheme does not require any complex operation such as error normalization and weight-threshold balancing, and can achieve an accuracy of around 97.5% on MNIST dataset using only 158,800 synapses. The multiplier-less inference engine trained using the proposed hard sigmoid SNN training (HaSiST) scheme can operate at a frequency of 135 MHz and consumes only 1.03 slice registers per synapse, 2.8 slice look-up tables, and can infer about 0.03$$\times {\varvec{10}}^{\varvec{9}}$$ × 10 9 features in a second, equivalent to 9.44 giga synaptic operations per second (GSOPS). The article also presents a high-speed, cost-efficient SNN training engine that consumes only 2.63 slice registers per synapse, 37.84 slice look-up tables per synapse, and can operate at a maximum computational frequency of around 50 MHz on a Virtex 6 FPGA.

show abstract

“…A typical example is that of 34 , where the researchers use a 38-layer ResNet. The scheme uses batch normalization through time (BNTT), instead of simple batchnorm to improve performance.…”

Section: Related Work and Problem Definitionmentioning

confidence: 99%

A low cost neuromorphic learning engine based on a high performance supervised SNN learning algorithm

Siddique

Vai

Pun

2023

Sci Rep

View full text Add to dashboard Cite

show abstract

“…During each iteration of the computation of the PNMS, the neuron ignition is determined jointly with the value of the feature and the issuance of the neighboring neuron pulses, and the issuance of the neighboring neuron pulses is associated with the structure of the spatial information; therefore, global temporal representation of spatial information can not only express its structure but also be used as spatial information feature. Based on the structural characteristics of neurons and their connections, the element of the global spatial information time representation of the system is the value (amount) of neuron ignition during each system iteration calculation, and the size of this value is related to the number of neurons, independent of the structure [26].…”

Section: Global Spatial Information Representationmentioning

confidence: 99%

Spatio-Temporal Coding-Based Helicopter Trajectory Planning for Pulsed Neural Membrane System

Huang

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

For the trajectory planning problem under the nonlinear and strongly coupled characteristics of unmanned helicopters, membrane computing with distributed parallel processing capability is introduced for unmanned helicopter trajectory planning. The global and local spatial information is temporally characterized; the temporal characterization algorithm under mapping information is designed; the hierarchical discriminant regression algorithm is designed based on incremental principal component analysis to realize the process of building and identifying trees in trajectory planning; and the pulsed neural membrane system (PNMS) with spatio-temporal coding function under membrane computing is constructed. Compared with the RRT algorithm in two experimental environments, the original path length, the trimmed path length, the time used to plan the trajectory, and the number of search nodes have different levels of improvement; the feasibility and effectiveness of the PNMS in unmanned helicopter trajectory planning are verified. It expands the theoretical research of membrane computing in the field of optimal control and provides theoretical support for the subsequent application practice.

show abstract

“…In recent years, spiking neural networks (SNNs) have emerged as a promising low-power alternative to ANNs. SNNs replicate the temporal and sparse spiking behavior exhibited by biological neurons (Roy et al, 2019;Davies et al, 2021;Manna et al, 2022Manna et al, , 2023Vicente-Sola et al, 2022). Unlike traditional neural networks, SNN neurons consume energy only during spike generation, leading to sparser activations and natural enhancements in Size, Weight, and Power (SWaP) characteristics.…”

Section: Introductionmentioning

confidence: 99%

Spiking neural networks fine-tuning for brain image segmentation

Yue,

Baltes,

Abuhajar

et al. 2023

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

IntroductionThe field of machine learning has undergone a significant transformation with the progress of deep artificial neural networks (ANNs) and the growing accessibility of annotated data. ANNs usually require substantial power and memory usage to achieve optimal performance. Spiking neural networks (SNNs) have recently emerged as a low-power alternative to ANNs due to their sparsity nature. Despite their energy efficiency, SNNs are generally more difficult to be trained than ANNs.MethodsIn this study, we propose a novel three-stage SNN training scheme designed specifically for segmenting human hippocampi from magnetic resonance images. Our training pipeline starts with optimizing an ANN to its maximum capacity, then employs a quick ANN-SNN conversion to initialize the corresponding spiking network. This is followed by spike-based backpropagation to fine-tune the converted SNN. In order to understand the reason behind performance decline in the converted SNNs, we conduct a set of experiments to investigate the output scaling issue. Furthermore, we explore the impact of binary and ternary representations in SNN networks and conduct an empirical evaluation of their performance through image classification and segmentation tasks.Results and discussionBy employing our hybrid training scheme, we observe significant advantages over both ANN-SNN conversion and direct SNN training solutions in terms of segmentation accuracy and training efficiency. Experimental results demonstrate the effectiveness of our model in achieving our design goals.

show abstract

Keys to accurate feature extraction using residual spiking neural networks

Cited by 12 publications

References 31 publications

A low cost neuromorphic learning engine based on a high performance supervised SNN learning algorithm

A low cost neuromorphic learning engine based on a high performance supervised SNN learning algorithm

Spatio-Temporal Coding-Based Helicopter Trajectory Planning for Pulsed Neural Membrane System

Spiking neural networks fine-tuning for brain image segmentation

Contact Info

Product

Resources

About