Learning target reaching motions with a robotic arm using brain-inspired dopamine modulated STDP

Tieck, J. Camilo Vasquez; Becker, Pascal; Kaiser, Jacques; Peric, Igor; Akl, Mahmoud; Reichard, Daniel; Roennau, Arne; Dillmann, Rüdiger

doi:10.1109/iccicc46617.2019.9146079

Cited by 5 publications

(2 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reinforcement Learning with SNNs has been previously explored through biological learning rules like reward-modulated STDP (R-STDP). An SNN was trained with R-STDP in [2] to make a vehicle keep its lane and in [33] for target reaching with a robotic arm. However, R-STDP only works with shallow networks and may not be as powerful as modern deep reinforcement learning methods in more complex tasks.…”

Section: Related Workmentioning

confidence: 99%

Porting Deep Spiking Q-Networks to neuromorphic chip Loihi

Akl¹,

Sandamirskaya

Walter³

et al. 2021

International Conference on Neuromorphic Systems 2021

Self Cite

View full text Add to dashboard Cite

Deep neural networks (DNNs) set the benchmark in many tasks in perception and control. Spiking versions of DNNs, implemented on neuromorphic hardware can enable orders of magnitude lower power consumption and low latency during network use. In this paper, we explore behavior and generalization capability of spiking, quantized spiking, and hardware implementation of deep Qnetworks in two classical reinforcement learning tasks. We found that spiking neural networks have slightly decreased performance compared to non-spiking network, but we can avoid performance degradation from quantization and in-chip implementation. We conclude that since hardware implementation leads to lower power consumption and low latency, neuromorphic approach is a promising avenue for deep Q-learning. Furthermore, online learning, enabled in neuromorphic chips, can be used to compensate for the performance decrease in environments with parameter variations. CCS CONCEPTS• Computing methodologies → Reinforcement learning.

show abstract

Section: Related Workmentioning

confidence: 99%

Porting Deep Spiking Q-Networks to neuromorphic chip Loihi

Akl¹,

Sandamirskaya

Walter³

et al. 2021

International Conference on Neuromorphic Systems 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…In some cases, this learning technique is not enough and it is necessary to apply reinforcement learning strategies too to improve the performance of the application. Examples of this are in the field of robotics: a line follower vehicle [1], snake-like robotics [3], obstacle avoidance [2] or target reaching motion with a robotic arm [18]. The authors of [13] proposed a method of R-STDP based on the combination of a neuromodulator (dopamine) and the signal produced by the STDP.…”

Section: Introductionmentioning

confidence: 99%

Bio-plausible digital implementation of a reward modulated STDP synapse

Quintana

Perez-Peña

Galindo

2022

Neural Comput & Applic

View full text Add to dashboard Cite

Reward-modulated Spike-Timing-Dependent Plasticity (R-STDP) is a learning method for Spiking Neural Network (SNN) that makes use of an external learning signal to modulate the synaptic plasticity produced by Spike-Timing-Dependent Plasticity (STDP). Combining the advantages of reinforcement learning and the biological plausibility of STDP, online learning on SNN in real-world scenarios can be applied. This paper presents a fully digital architecture, implemented on an Field-Programmable Gate Array (FPGA), including the R-STDP learning mechanism in a SNN. The hardware results obtained are comparable to the software simulations results using the Brian2 simulator. The maximum error is of 0.083 when a 14-bits fix-point precision is used in realtime. The presented architecture shows an accuracy of 95% when tested in an obstacle avoidance problem on mobile robotics with a minimum use of resources.

show abstract

An Imbalanced R-STDP Learning Rule in Spiking Neural Networks for Medical Image Classification

Zhou

Ren

2020

IEEE Access

View full text Add to dashboard Cite

Spiking neural networks (SNNs) have the advantages of inherent power-efficiency, biological plausibility and good image recognition performance. They are good candidates for medical image classification especially when the labeled training data are limited. In medical image classification, one of the major challenges is the highly class imbalanced problem which causes deep learning networks to bias towards the majority class and poorly recognizes the minority class. Despite that there are some methods for addressing this problem, very few algorithm-level methods exist for SNNs. In this work, we propose an imbalanced reward-modulation spike-timing-dependent plasticity (R-STDP) learning rule for SNNs to solve the medical image class imbalanced problem. We introduce an imbalanced reward coefficient for the R-STDP learning rule to set the reward from the minority class to be higher than that of the majority class, and this reward coefficient can help to set the class-dependent rewards according to the data statistic of the training dataset. Experiment results on three benchmark datasets with imbalanced splits show that our method significantly improves the performance than that of the baseline SNNs and outperforms the compared state-of-the-art methods addressing medical image class imbalanced problem including datalevel and algorithm-level methods. Moreover, our method achieves excellent classification performance on the imbalanced medical dataset ISIC-2018. The results show that the proposed method can well help SNNs in classifying imbalanced medical image datasets. Besides, our proposed method can obtain high sensitivity to disease class by adjusting the reward coefficient, which is very useful for identifying disease samples in medical diagnostic tasks.INDEX TERMS Class imbalance, spiking neural networks (SNNs), reward-modulated spike-timingdependent plasticity (R-STDP), medical image.

show abstract

Learning target reaching motions with a robotic arm using brain-inspired dopamine modulated STDP

Cited by 5 publications

References 20 publications

Porting Deep Spiking Q-Networks to neuromorphic chip Loihi

Porting Deep Spiking Q-Networks to neuromorphic chip Loihi

Bio-plausible digital implementation of a reward modulated STDP synapse

An Imbalanced R-STDP Learning Rule in Spiking Neural Networks for Medical Image Classification

Contact Info

Product

Resources

About