Combining backpropagation with Equilibrium Propagation to improve an Actor-Critic reinforcement learning framework

Kubo, Yoshimasa; Chalmers, Eric; Luczak, Artur

doi:10.3389/fncom.2022.980613

Cited by 3 publications

(3 citation statements)

References 29 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MLP continuously learns to produce increasingly accurate predictions by repeatedly modifying the weights depending on the computed gradients. 99 BP-MLP has also achieved 81% accuracy for their prediction model for four tasters' parameters (leaf quality, infusion, liquor, and aroma) used for their black tea samples to correlate with sensory analysis responses, which was lower than the 90% obtained with the PNN. 98 According to the authors, greater PNN values might be linked to their dataset size and also the idea that the PNN is derived from a probability base that uses a previous sample to move forward the model output, which depending on the variations in your sample size favors more accurate values.…”

Section: Data Treatment Used For Lc-e-nose Based On Machine Learningmentioning

confidence: 87%

Low-cost electronic-nose (LC-e-nose) systems for the evaluation of plantation and fruit crops: recent advances and future trends

Vinicius da Silva Ferreira,

Barbosa,

Kamruzzaman

et al. 2023

Anal. Methods

View full text Add to dashboard Cite

show abstract

Section: Data Treatment Used For Lc-e-nose Based On Machine Learningmentioning

confidence: 87%

Low-cost electronic-nose (LC-e-nose) systems for the evaluation of plantation and fruit crops: recent advances and future trends

Vinicius da Silva Ferreira,

Barbosa,

Kamruzzaman

et al. 2023

Anal. Methods

View full text Add to dashboard Cite

show abstract

“…In future work, we plan to apply models with adaptation to neuronal data analyses [ 43–47 ] and to reinforcement learning tasks [ 48 , 49 ]. Regularization effect of adaptation may help to improve training networks with BP.…”

Section: Discussionmentioning

confidence: 99%

Biologically-inspired neuronal adaptation improves learning in neural networks

Kubo

Chalmers

Luczak

2023

Communicative & Integrative Biology

Self Cite

View full text Add to dashboard Cite

Since humans still outperform artificial neural networks on many tasks, drawing inspiration from the brain may help to improve current machine learning algorithms. Contrastive Hebbian learning (CHL) and equilibrium propagation (EP) are biologically plausible algorithms that update weights using only local information (without explicitly calculating gradients) and still achieve performance comparable to conventional backpropagation. In this study, we augmented CHL and EP with Adjusted Adaptation , inspired by the adaptation effect observed in neurons, in which a neuron’s response to a given stimulus is adjusted after a short time. We add this adaptation feature to multilayer perceptrons and convolutional neural networks trained on MNIST and CIFAR-10. Surprisingly, adaptation improved the performance of these networks. We discuss the biological inspiration for this idea and investigate why Neuronal Adaptation could be an important brain mechanism to improve the stability and accuracy of learning.

show abstract

“…These agents actively engage with their surroundings, with deep neural networks guiding their decisions and actions ( Osband et al, 2016 ). Among the various algorithms in DRL, the Actor-Critic (AC) algorithm is recognized as a prominent and effective approach ( Kubo, Chalmers & Luczak, 2022 ).…”

Section: Introductionmentioning

confidence: 99%

A priority experience replay actor-critic algorithm using self-attention mechanism for strategy optimization of discrete problems

Sun,

Yang

2024

PeerJ Computer Science

View full text Add to dashboard Cite

In the dynamic field of deep reinforcement learning, the self-attention mechanism has been increasingly recognized. Nevertheless, its application in discrete problem domains has been relatively limited, presenting complex optimization challenges. This article introduces a pioneering deep reinforcement learning algorithm, termed Attention-based Actor-Critic with Priority Experience Replay (A2CPER). A2CPER combines the strengths of self-attention mechanisms with the Actor-Critic framework and prioritized experience replay to enhance policy formulation for discrete problems. The algorithm’s architecture features dual networks within the Actor-Critic model—the Actor formulates action policies and the Critic evaluates state values to judge the quality of policies. The incorporation of target networks aids in stabilizing network optimization. Moreover, the addition of self-attention mechanisms bolsters the policy network’s capability to focus on critical information, while priority experience replay promotes training stability and reduces correlation among training samples. Empirical experiments on discrete action problems validate A2CPER’s adeptness at policy optimization, marking significant performance improvements across tasks. In summary, A2CPER highlights the viability of self-attention mechanisms in reinforcement learning, presenting a robust framework for discrete problem-solving and potential applicability in complex decision-making scenarios.

show abstract

Combining backpropagation with Equilibrium Propagation to improve an Actor-Critic reinforcement learning framework

Cited by 3 publications

References 29 publications

Low-cost electronic-nose (LC-e-nose) systems for the evaluation of plantation and fruit crops: recent advances and future trends

Low-cost electronic-nose (LC-e-nose) systems for the evaluation of plantation and fruit crops: recent advances and future trends

Biologically-inspired neuronal adaptation improves learning in neural networks

A priority experience replay actor-critic algorithm using self-attention mechanism for strategy optimization of discrete problems

Contact Info

Product

Resources

About