Mastering the Output Frequency in Spiking Neural Networks

Falez, Pierre; Tirilly, Pierre; Bilasco, Ioan Marius; Devienne, Philippe; Boulet, Pierre

doi:10.1109/ijcnn.2018.8489410

Cited by 13 publications

(8 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An exponential leak is applied to help neurons with weak activities. However, this mechanism uses two parameters, which makes the search of suited values difficult [13]. Moreover, those parameters do not enhance the convergence towards the different types of patterns shown in Figure 2.…”

Section: A Time Target Threshold Adaptationmentioning

confidence: 99%

Multi-layered Spiking Neural Network with Target Timestamp Threshold Adaptation and STDP

Falez

Tirilly

Bilasco

et al. 2019

2019 International Joint Conference on Neural Networks (IJCNN)

Self Cite

View full text Add to dashboard Cite

Spiking neural networks (SNNs) are good candidates to produce ultra-energy-efficient hardware. However, the performance of these models is currently behind traditional methods. Introducing multi-layered SNNs is a promising way to reduce this gap. We propose in this paper a new threshold adaptation system which uses a timestamp objective at which neurons should fire. We show that our method leads to state-of-the-art classification rates on the MNIST dataset (98.60%) and the Faces/Motorbikes dataset (99.46%) with an unsupervised SNN followed by a linear SVM. We also investigate the sparsity level of the network by testing different inhibition policies and STDP rules.

show abstract

Section: A Time Target Threshold Adaptationmentioning

confidence: 99%

Multi-layered Spiking Neural Network with Target Timestamp Threshold Adaptation and STDP

Falez

Tirilly

Bilasco

et al. 2019

2019 International Joint Conference on Neural Networks (IJCNN)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Computational neuroscience has provided a new idea for unsupervised learning mechanisms. Spiking Time Dependent Plasticity (STDP) (Abbott and Nelson, 2000;Song et al, 2000;Caporale and Dan, 2008;Tavanaei et al, 2018;Falez et al, 2019), is a temporally asymmetric form of Hebbian learning and is the most widely used unsupervised learning mechanism in SNN. In the temporal dimension, the relation between the presynaptic action potential and the postsynaptic action potential regulates the neurons' weights, which is a feature unique to SNN.…”

Section: A New Approach Of Unsupervised Learningmentioning

confidence: 99%

Linear leaky-integrate-and-fire neuron model based spiking neural networks and its mapping relationship to deep neural networks

Lü

Xing

2022

Front. Neurosci.

View full text Add to dashboard Cite

Spiking neural networks (SNNs) are brain-inspired machine learning algorithms with merits such as biological plausibility and unsupervised learning capability. Previous works have shown that converting Artificial Neural Networks (ANNs) into SNNs is a practical and efficient approach for implementing an SNN. However, the basic principle and theoretical groundwork are lacking for training a non-accuracy-loss SNN. This paper establishes a precise mathematical mapping between the biological parameters of the Linear Leaky-Integrate-and-Fire model (LIF)/SNNs and the parameters of ReLU-AN/Deep Neural Networks (DNNs). Such mapping relationship is analytically proven under certain conditions and demonstrated by simulation and real data experiments. It can serve as the theoretical basis for the potential combination of the respective merits of the two categories of neural networks.

show abstract

“…We observed that during the learning process, some neurons fail to learn a specific class, which affects the network performance resulting in false classifications. On the other hand, applying Progressive Pruning on the network results in a decrease of the average spiking network frequency, which will affect the energy consumption of our network positively, but may cause a frequency loss in multilayer networks, which is a known issue in Convolutional Spiking Neural Networks (CSNN) as described in [24]. Based on those observations, we propose a Dynamic Synaptic Weight Reinforcement (DSWR), which concerns the synapses that are conserved and considered as critical, to improve the network performance, by pushing the neurons that did not specialize in a specific pattern or class to do so, and keep the average spiking frequency of the network near the baseline.…”

Section: B Dynamic Synaptic Weight Reinforcementmentioning

confidence: 99%

“…Applying PP on the network results in a decrease in the average spiking network frequency, which will affect the energy consumption of our network positively. However, it may cause a frequency loss in multilayer networks, which is a known issue in convolutional spiking neural networks (CSNNs) as described in Reference 37.…”

Section: Contributionmentioning

confidence: 99%

Progressive compression and weight reinforcement for spiking neural networks

Elbez

Benhaoua

Devienne

et al. 2022

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

Neuromorphic architectures are one of the most promising architectures to significantly reduce the energy consumption of tomorrow's computers. These architectures are inspired by the behaviour of the brain at a fairly precise level and consist of artificial Spiking Neural Networks (SNNs). To optimise the implementation of these architectures, we propose in this paper a novel progressive network compression and reinforcement technique based on two functions, progressive pruning and dynamic synaptic weight reinforcement used after each training batch. The proposed approach delivers a highly compressed network (up to 75 % of compression rate) while preserving the network performance when tested with MNIST.

show abstract

Mastering the Output Frequency in Spiking Neural Networks

Cited by 13 publications

References 25 publications

Multi-layered Spiking Neural Network with Target Timestamp Threshold Adaptation and STDP

Multi-layered Spiking Neural Network with Target Timestamp Threshold Adaptation and STDP

Linear leaky-integrate-and-fire neuron model based spiking neural networks and its mapping relationship to deep neural networks

Progressive compression and weight reinforcement for spiking neural networks

Contact Info

Product

Resources

About