Efficacy of Modern Neuro-Evolutionary Strategies for Continuous Control Optimization

Pagliuca, Paolo; Milano, Nicola; Nolfi, Stefano

doi:10.3389/frobt.2020.00098

Cited by 31 publications

(36 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More specifically, the length and the mass of the second pole corresponds to # $ of the length of the first pole (instead of # #: ), and the range of variation of the environmental conditions is much larger than in the classic double-pole balancing problem. These variations increase significantly the complexity of the problem (for more details see Pagliuca, Milano andNolfi, 2018 andPagliuca andNolfi, 2019).…”

Section: The Adaptive Problemsmentioning

confidence: 99%

Automated curriculum learning for embodied agents a neuroevolutionary approach

Milano

Nolfi

2021

Sci Rep

Self Cite

View full text Add to dashboard Cite

We demonstrate how the evolutionary training of embodied agents can be extended with a curriculum learning algorithm that automatically selects the environmental conditions in which the evolving agents are evaluated. The environmental conditions are selected to adjust the level of difficulty to the ability level of the current evolving agents, and to challenge the weaknesses of the evolving agents. The method does not require domain knowledge and does not introduce additional hyperparameters. The results collected on two benchmark problems, that require to solve a task in significantly varying environmental conditions, demonstrate that the method proposed outperforms conventional learning methods and generates solutions which are robust to variations and able to cope with different environmental conditions.

show abstract

Section: The Adaptive Problemsmentioning

confidence: 99%

Automated curriculum learning for embodied agents a neuroevolutionary approach

Milano

Nolfi

2021

Sci Rep

Self Cite

View full text Add to dashboard Cite

show abstract

“…The connection weights and the biases of the control network are trained through the Open-AI evolutionary strategy [ 1 ], i.e. one of the most effective evolutionary methods for continuous problem optimization [ 2 ]. Agents are evaluated for 1 episode in the case of the Walker2DBullet, HalfCheetahBullet, BipedalWalkerHardcore, and MIT racecar and for 5 episodes in the case of the BipedalWalkerHardcore.…”

Section: Experimental Settingmentioning

confidence: 99%

Autonomous learning of features for control: Experiments with embodied and situated agents

Milano

Nolfi

2021

PLoS ONE

Self Cite

View full text Add to dashboard Cite

The efficacy of evolutionary or reinforcement learning algorithms for continuous control optimization can be enhanced by including an additional neural network dedicated to features extraction trained through self-supervision. In this paper we introduce a method that permits to continue the training of the features extracting network during the training of the control network. We demonstrate that the parallel training of the two networks is crucial in the case of agents that operate on the basis of egocentric observations and that the extraction of features provides an advantage also in problems that do not benefit from dimensionality reduction. Finally, we compare different feature extracting methods and we show that sequence-to-sequence learning outperforms the alternative methods considered in previous studies.

show abstract

“…Gaming similarly to DRL, gaming represents one of the main testbeds for ES. Most of the literature on ES reviewed in this survey test their algorithms on Atari games [4,5,29,32,256,257,258,259]. These are considered to be challenging as they present the agents with high dimensional visual inputs and a diverse and interesting set of tasks that were designed to be difficult for humans players [14].…”

Section: B Evolution Strategy Applicationsmentioning

confidence: 99%

Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey

Majid¹

2021

Preprint

View full text Add to dashboard Cite

<div>Deep Reinforcement Learning (DRL) has the potential to surpass human-level control in sequential decision-making problems. Evolution Strategies (ESs) have different characteristics than DRL, yet they are promoted as a scalable alternative. </div><div>To get insights into their strengths and weaknesses, in this paper, we put the two approaches side by side. After presenting the fundamental concepts and algorithms for each of the two approaches, they are compared from the perspectives of scalability, exploration, adaptation to dynamic environments, and multi-agent learning. Then, the paper discusses hybrid algorithms, combining aspects of both DRL and ESs, and how they attempt to capitalize on the benefits of both techniques. Lastly, both approaches are compared based on the set of applications they support, showing their potential for tackling real-world problems.</div><div>This paper aims to present an overview of how DRL and ESs can be used, either independently or in unison, to solve specific learning tasks. It is intended to guide researchers to select which method suits them best and provides a bird's eye view of the overall literature in the field. Further, we also provide application scenarios and open challenges. </div>

show abstract

Efficacy of Modern Neuro-Evolutionary Strategies for Continuous Control Optimization

Cited by 31 publications

References 30 publications

Automated curriculum learning for embodied agents a neuroevolutionary approach

Automated curriculum learning for embodied agents a neuroevolutionary approach

Autonomous learning of features for control: Experiments with embodied and situated agents

Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey

Contact Info

Product

Resources

About