First-Spike-Based Visual Categorization Using Reward-Modulated STDP

Mozafari, Milad; Kheradpisheh, Saeed Reza; Masquelier, Timothée; Nowzari-Dalini, Abbas; Ganjtabesh, Mohammad

doi:10.1109/tnnls.2018.2826721

Cited by 150 publications

(133 citation statements)

References 89 publications

Supporting

Mentioning

127

Contrasting

Order By: Relevance

“…However, to have a fully hardware-implementable SNN, using bioinspired classifiers is required. Recent work succeeds in using supervised STDP as a classifier in multi-layered SNN [23]. We aim at investigating the performance of our model with such learning rules, while respecting the constraint of local computations.…”

Section: Faces/motorbikesmentioning

confidence: 99%

Multi-layered Spiking Neural Network with Target Timestamp Threshold Adaptation and STDP

Falez

Tirilly

Bilasco

et al. 2019

2019 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Spiking neural networks (SNNs) are good candidates to produce ultra-energy-efficient hardware. However, the performance of these models is currently behind traditional methods. Introducing multi-layered SNNs is a promising way to reduce this gap. We propose in this paper a new threshold adaptation system which uses a timestamp objective at which neurons should fire. We show that our method leads to state-of-the-art classification rates on the MNIST dataset (98.60%) and the Faces/Motorbikes dataset (99.46%) with an unsupervised SNN followed by a linear SVM. We also investigate the sparsity level of the network by testing different inhibition policies and STDP rules.

show abstract

Section: Faces/motorbikesmentioning

confidence: 99%

Multi-layered Spiking Neural Network with Target Timestamp Threshold Adaptation and STDP

Falez

Tirilly

Bilasco

et al. 2019

2019 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

show abstract

“…R-STDP is only applied to the last layer in Task 1. As we showed in Task 2 and our previous work [45], R-STDP can be better than STDP when computational resources are limited. In Task 1, the goal was to increase the performance of the network to an acceptable and competing level.…”

Section: Discussionmentioning

confidence: 59%

“…The proposed network inherits bio-plausability, energy efficiency and hardware friendliness from its ancestors [34,45], but is now able to solve harder and complex recognition tasks thanks to its deeper structure. Our network has two fundamental features that elevate its energy efficacy in comparison to the other deep networks, particularly DCNNs: (1) Communication with spikes: Using spike/nospike regime, the accumulation of the incoming spikes can be implemented by energy-efficient "accumulator" units.…”

Section: Discussionmentioning

confidence: 99%

“…Conversely, the release of ACh can signal punishment and result in discouraging the neuron's behavior [47]. For differences between STDP and R-STDP in action, we refer the readers to [45].…”

Section: Plasticitymentioning

confidence: 99%

“…We previously developed a shallow SNN with a single trainable layer [45], where the plasticity was governed by reward-modulated STDP (R-STDP). R-STDP is a reinforcement learning rule inspired by the roles of neuromodulators such as Dopamine (DA) and Acetylcholine (ACh) in modulation of STDP [46,47].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks

Mozafari

Ganjtabesh

Nowzari-Dalini

et al. 2019

Pattern Recognition

Self Cite

134

View full text Add to dashboard Cite

The primate visual system has inspired the development of deep artificial neural networks, which have revolutionized the computer vision domain. Yet these networks are much less energy-efficient than their biological counterparts, and they are typically trained with backpropagation, which is extremely data-hungry. To address these limitations, we used a deep convolutional spiking neural network (DCSNN) and a latency-coding scheme. We trained it using a combination of spike-timingdependent plasticity (STDP) for the lower layers and reward-modulated STDP (R-STDP) for the higher ones. In short, with R-STDP a correct (resp. incorrect) decision leads to STDP (resp. anti-STDP). This approach led to an accuracy of 97.2% on MNIST, without requiring an external classifier. In addition, we demonstrated that R-STDP extracts features that are diagnostic for the task at hand, and discards the other ones, whereas STDP extracts any feature that repeats. Finally, our ap- * Corresponding author.Email addresses: milad.mozafari@ut.ac.ir (MM), mgtabesh@ut.ac.ir (MG) nowzari@ut.ac.ir (AND) simon.thorpe@cnrs.fr (SJT) timothee.masquelier@cnrs.fr (TM).proach is biologically plausible, hardware friendly, and energy-efficient.

show abstract

A Reconfigurable Two‐WSe₂‐Transistor Synaptic Cell for Reinforcement Learning

et al. 2022

View full text Add to dashboard Cite

learning, reinforcement learning (RL) is one of machine learning paradigms, which allow the machine to interact with the environment and update the policy according to the negative or positive reward signals. [10,11] The algorithms of RL [10] mainly include artificial neural network (ANN)-based deep Q-learning (DQN) and spiking neural network (SNN)based reward-modulated spike-timingdependent plasticity (R-STDP). Compared with ANN-based DQN that adopts error backpropagation and gradient descent to update the weight, R-STDP updates the weight by incorporating the brain-inspired STDP with reward signals being biologically more plausible. [12][13][14][15][16] The hardware implementation of RL paradigm with DQN based on emerging nonvolatile memory technologies (NVMs) has been reported recently. [17][18][19] However, hardware realization of R-STDP based on NVMs is far more less explored. The R-STDP learning rule mainly relies on the modulation of the "standard" STDP by a reward term (positive or negative reward), which is from the external environment, thus realizing SNN-based reinforcement learning. R-STDP stores the traces of synapses that are eligible for STDP (eligibility traces), and applies the modulated weight changes at the time of receiving a positive/negative reward signal. Eligibility traces are a critical ingredient of R-STDP learning rules. [20] There have been some works realizing long-lasting eligibility traces with complementary metal-oxide-semiconductor (CMOS) technology combining large capacitors. [21][22][23][24][25] Recently, Demirağ et al. exploit the drift behavior of phase change memory devices to intrinsically perform eligibility traces with long timescales realizing higher area efficiency than the CMOS ones. [26] In addition to realize eligibility traces constructing STDP, the polarity of STDP will be further changed according to the sign of external reward signal. Thus, it requires emerging nonvolatile devices with reconfigurable characteristics for both STDP and anti-STDP.2D semiconductor field-effect (FE) transistors (FETs) with atomic-level thickness show excellent electrostatic tunability, [27][28][29][30][31][32][33] which provides the possibility of modulating the channel to be reconfigurable p-type or n-type for both STDP and anti-STDP learning rules. On the other hand, to realize nonvolatile memory characteristics, we adopt the ferroelectric poly(vinylidene fluoride-trifluoroethylene) (P(VDF-TrFE)) as gate dielectric for dynamically tunable memory Reward-modulated spike-timing-dependent plasticity (R-STDP) is a braininspired reinforcement learning (RL) rule, exhibiting potential for decisionmaking tasks and artificial general intelligence. However, the hardware implementation of the reward-modulation process in R-STDP usually requires complicated Si complementary metal-oxide-semiconductor (CMOS) circuit design that causes high power consumption and large footprint. Here, a design with two synaptic transistors (2T) connected in a parallel structure is experimentally demonstrated. The 2T...

show abstract

First-Spike-Based Visual Categorization Using Reward-Modulated STDP

Cited by 150 publications

References 89 publications

Multi-layered Spiking Neural Network with Target Timestamp Threshold Adaptation and STDP

Multi-layered Spiking Neural Network with Target Timestamp Threshold Adaptation and STDP

Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks

A Reconfigurable Two‐WSe₂‐Transistor Synaptic Cell for Reinforcement Learning

Contact Info

Product

Resources

About

First-Spike-Based Visual Categorization Using Reward-Modulated STDP

Cited by 150 publications

References 89 publications

Multi-layered Spiking Neural Network with Target Timestamp Threshold Adaptation and STDP

Multi-layered Spiking Neural Network with Target Timestamp Threshold Adaptation and STDP

Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks

A Reconfigurable Two‐WSe2‐Transistor Synaptic Cell for Reinforcement Learning

Contact Info

Product

Resources

About

A Reconfigurable Two‐WSe₂‐Transistor Synaptic Cell for Reinforcement Learning