A Hebbian-Based Reinforcement Learning Framework for Spike-Timing-Dependent Synapses

El-Laithy, Karim; Bogdan, Martin

doi:10.1007/978-3-642-15822-3_21

Cited by 3 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that in the lower panel of (c), the length of F (or G ) is 200/ 𝒲 , where 𝒲 = 5. Both (b) and (c) are adopted from [ 12 ].…”

Section: Figurementioning

confidence: 99%

“…In the study at hand, we propose a follow-up study to the introductory framework introduced in [12]. (The results reported here are separately produced and not adopted from [12]. )…”

Section: Introductionmentioning

confidence: 99%

“…It is tempting, therefore, to investigate whether the Hebbian/anti-Hebbian modulation of STDP within an RL framework, that is, with a reward signal, can lead to RL when the learning is directed to tune the hidden parameters of a synaptic model. In the study at hand, we propose a follow-up study to the introductory framework introduced in [ 12 ]. (The results reported here are separately produced and not adopted from [ 12 ].)…”

Section: Introductionmentioning

confidence: 99%

“…In the study at hand, we propose a follow-up study to the introductory framework introduced in [ 12 ]. (The results reported here are separately produced and not adopted from [ 12 ].) The framework integrates the concepts of both Hebbian/anti-Hebbian learning and RL while explicitly using plausible biological neuronal and synaptic representations.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Reinforcement Learning Framework for Spiking Networks with Dynamic Synapses

El-Laithy

Bogdan

2011

Computational Intelligence and Neuroscience

Self Cite

View full text Add to dashboard Cite

An integration of both the Hebbian-based and reinforcement learning (RL) rules is presented for dynamic synapses. The proposed framework permits the Hebbian rule to update the hidden synaptic model parameters regulating the synaptic response rather than the synaptic weights. This is performed using both the value and the sign of the temporal difference in the reward signal after each trial. Applying this framework, a spiking network with spike-timing-dependent synapses is tested to learn the exclusive-OR computation on a temporally coded basis. Reward values are calculated with the distance between the output spike train of the network and a reference target one. Results show that the network is able to capture the required dynamics and that the proposed framework can reveal indeed an integrated version of Hebbian and RL. The proposed framework is tractable and less computationally expensive. The framework is applicable to a wide class of synaptic models and is not restricted to the used neural representation. This generality, along with the reported results, supports adopting the introduced approach to benefit from the biologically plausible synaptic models in a wide range of intuitive signal processing.

show abstract

“…Note that in the lower panel of (c), the length of F (or G ) is 200/ 𝒲 , where 𝒲 = 5. Both (b) and (c) are adopted from [ 12 ].…”

Section: Figurementioning

confidence: 99%

“…In the study at hand, we propose a follow-up study to the introductory framework introduced in [12]. (The results reported here are separately produced and not adopted from [12]. )…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Reinforcement Learning Framework for Spiking Networks with Dynamic Synapses

El-Laithy

Bogdan

2011

Computational Intelligence and Neuroscience

Self Cite

View full text Add to dashboard Cite

show abstract

A Hypothetical Free Synaptic Energy Function and Related States of Synchrony

El-Laithy

Bogdan

2011

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Reinforcement Learning in Spiking Neural Networks with Stochastic and Deterministic Synapses

Yuan

Yan

et al. 2019

Neural Computation

View full text Add to dashboard Cite

Though succeeding in solving various learning tasks, most existing reinforcement learning (RL) models have failed to take into account the complexity of synaptic plasticity in the neural system. Models implementing reinforcement learning with spiking neurons involve only a single plasticity mechanism. Here, we propose a neural realistic reinforcement learning model that coordinates the plasticities of two types of synapses: stochastic and deterministic. The plasticity of the stochastic synapse is achieved by the hedonistic rule through modulating the release probability of synaptic neurotransmitter, while the plasticity of the deterministic synapse is achieved by a variant of a reward-modulated spike-timing-dependent plasticity rule through modulating the synaptic strengths. We evaluate the proposed learning model on two benchmark tasks: learning a logic gate function and the 19-state random walk problem. Experimental results show that the coordination of diverse synaptic plasticities can make the RL model learn in a rapid and stable form.

show abstract

A Hebbian-Based Reinforcement Learning Framework for Spike-Timing-Dependent Synapses

Cited by 3 publications

References 21 publications

A Reinforcement Learning Framework for Spiking Networks with Dynamic Synapses

A Reinforcement Learning Framework for Spiking Networks with Dynamic Synapses

A Hypothetical Free Synaptic Energy Function and Related States of Synchrony

Reinforcement Learning in Spiking Neural Networks with Stochastic and Deterministic Synapses

Contact Info

Product

Resources

About