Hybridizing novelty search for transfer learning

Didi, Sabre; Nitschke, Geoff

doi:10.1109/ssci.2016.7850180

Cited by 9 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Keepaway is a subtask of RoboCup that was put forth as a testbed for machine learning in 2001 (Stone & Sutton, 2001). It has since been used for research on temporal difference reinforcement learning with function approximation (Stone, Sutton, & Kuhlmann, 2005), evolutionary learning (Pietro et al, 2002), relational reinforcement learning (Walker et al, 2004), behaviour transfer (Cheng et al, 2018;Didi & Nitschke, 2016a, 2016bNitschke & Didi, 2017;Schwab et al, 2018;, batch reinforcement learning (Riedmiller et al, 2009) and hierarchical reinforcement learning (Bai & Russell, 2017).…”

Section: Robocup Keepaway Soccermentioning

confidence: 99%

A survey of benchmarks for reinforcement learning algorithms

Stapelberg

Malan

2020

SACJ

View full text Add to dashboard Cite

Reinforcement learning has recently experienced increased prominence in the machine learning community. There are many approaches to solving reinforcement learning problems with new techniques developed constantly. When solving problems using reinforcement learning, there are various difficult challenges to overcome. \par To ensure progress in the field, benchmarks are important for testing new algorithms and comparing with other approaches. The reproducibility of results for fair comparison is therefore vital in ensuring that improvements are accurately judged. This paper provides an overview of different contributions to reinforcement learning benchmarking and discusses how they can assist researchers to address the challenges facing reinforcement learning. The contributions discussed are the most used and recent in the literature. The paper discusses the contributions in terms of implementation, tasks and provided algorithm implementations with benchmarks. \par The survey aims to bring attention to the wide range of reinforcement learning benchmarking tasks available and to encourage research to take place in a standardised manner. Additionally, this survey acts as an overview for researchers not familiar with the different tasks that can be used to develop and test new reinforcement learning algorithms.

show abstract

Section: Robocup Keepaway Soccermentioning

confidence: 99%

A survey of benchmarks for reinforcement learning algorithms

Stapelberg

Malan

2020

SACJ

View full text Add to dashboard Cite

show abstract

“…Keepaway is a subtask of RoboCup that was put forth as a testbed for machine learning in 2001 [17]. It has since been used for research on temporal difference reinforcement learning with function approximation [46], evolutionary learning [47], relational reinforcement learning [48], behaviour transfer [49,50,51,52,53,54,55], batch reinforcement learning [56] and hierarchical reinforcement learning [57].…”

Section: Robocup Keepaway Soccermentioning

confidence: 99%

A survey of benchmarking frameworks for reinforcement learning

Stapelberg,

Malan

2020

Preprint

View full text Add to dashboard Cite

Reinforcement learning has recently experienced increased prominence in the machine learning community. There are many approaches to solving reinforcement learning problems with new techniques developed constantly. When solving problems using reinforcement learning, there are various difficult challenges to overcome.To ensure progress in the field, benchmarks are important for testing new algorithms and comparing with other approaches. The reproducibility of results for fair comparison is therefore vital in ensuring that improvements are accurately judged. This paper provides an overview of different contributions to reinforcement learning benchmarking and discusses how they can assist researchers to address the challenges facing reinforcement learning. The contributions discussed are the most used and recent in the literature. The paper discusses the contributions in terms of implementation, tasks and provided algorithm implementations with benchmarks.The survey aims to bring attention to the wide range of reinforcement learning benchmarking tasks available and to encourage research to take place in a standardised manner. Additionally, this survey acts as an overview for researchers not familiar with the different tasks that can be used to develop and test new reinforcement learning algorithms.

show abstract

“…Furthermore, related work intersecting the fields of evolutionary controller design and policy (behavior) transfer 1 indicate that coupling evolutionary search with the transfer of behaviors between tasks of increasing complexity is an effective means to boost evolved behavior quality for a broad range of tasks Taylor et al, 2010;Verbancsics and Stanley, 2010;Didi and Nitschke, 2016a). Policy transfer is a method that aims to improve learning by leveraging knowledge from learning in related but simpler tasks (Pan and Yang, 2010).…”

Section: Introductionmentioning

confidence: 99%

Evolutionary Policy Transfer and Search Methods for Boosting Behavior Quality: RoboCup Keep-Away Case Study

Nitschke

Didi

2017

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

This study evaluates various evolutionary search methods to direct neural controller evolution in company with policy (behavior) transfer across increasingly complex collective robotic (RoboCup keep-away) tasks. Robot behaviors are first evolved in a source task and then transferred for further evolution to more complex target tasks. Evolutionary search methods tested include objective-based search (fitness function), behavioral and genotypic diversity maintenance, and hybrids of such diversity maintenance and objective-based search. Evolved behavior quality is evaluated according to effectiveness and efficiency. Effectiveness is the average task performance of transferred and evolved behaviors, where task performance is the average time the ball is controlled by a keeper team. Efficiency is the average number of generations taken for the fittest evolved behaviors to reach a minimum task performance threshold given policy transfer. Results indicate that policy transfer coupled with hybridized evolution (behavioral diversity maintenance and objective-based search) addresses the bootstrapping problem for increasingly complex keep-away tasks. That is, this hybrid method (coupled with policy transfer) evolves behaviors that could not otherwise be evolved. Also, this hybrid evolutionary search was demonstrated as consistently evolving topologically simple neural controllers that elicited high-quality behaviors.

show abstract

Hybridizing novelty search for transfer learning

Cited by 9 publications

References 25 publications

A survey of benchmarks for reinforcement learning algorithms

A survey of benchmarks for reinforcement learning algorithms

A survey of benchmarking frameworks for reinforcement learning

Evolutionary Policy Transfer and Search Methods for Boosting Behavior Quality: RoboCup Keep-Away Case Study

Contact Info

Product

Resources

About