Devising Effective Novelty Search Algorithms

Abstract

Novelty search is a state-of-the-art evolutionary approach that promotes behavioural novelty instead of pursuing a static objective. Along with a large number of successful applications, many different variants of novelty search have been proposed. It is still unclear, however, how some key parameters and algorithmic components influence the evolutionary dynamics and performance of novelty search. In this paper, we conduct a comprehensive empirical study focused on novelty search's algorithmic components. We study the k parameter -the number of nearest neighbours used in the computation of novelty scores; the use and function of an archive; how to combine novelty search with fitness-based evolution; and how to configure the mutation rate of the underlying evolutionary algorithm. Our study is conducted in a simulated maze navigation task. Our results show that the configuration of novelty search can have a significant impact on performance and behaviour space exploration. We conclude with a number of guidelines for the implementation and configuration of novelty search, which should help future practitioners to apply novelty search more effectively.

On the Critical Role of Divergent Selection in Evolvability

“…Thus there is consistent pressure to diverge, meaning mechanisms that enhance the ability to diverge can hitchhike along with the divergences they enable. Furthermore, because divergent algorithms like novelty search and MAP-Elites are designed to simultaneously maintain many distinct lineages (Lehman et al, 2012;Gomes et al, 2015;Nguyen et al, 2015), evolvable lineages can therefore distinguish themselves by consistently producing novelty. In this way, divergent selection systematically aligns with evolvability.…”

Evolutionary Policy Transfer and Search Methods for Boosting Behavior Quality: RoboCup Keep-Away Case Study

“…Specifically, replacing objective search with the search for behavioral diversity in controller evolution (Moriguchi and Honiden, 2010;Mouret and Doncieux, 2012;Lehman et al, 2013;Gomes et al, 2015) has been demonstrated to boost the quality of evolved behaviors across a range of simulated (Lehman and Stanley, 2011a;Mouret and Doncieux, 2012;Gomes et al, 2016) and physical (Cully et al, 2015;Cully and Mouret, 2016) ER tasks.…”

“…However, current empirical data indicate that for controller evolution to solve complex collective behavior tasks, then neither objective nor nonobjective-based search performs well (evolves high-quality behaviors). Rather, recent research results indicate that hybridizing these two search approaches facilitates the evolution of the high-quality behaviors (Gomes and Christensen, 2013b;Gomes et al, 2013Gomes et al, , 2015.…”

“…While the benefits of non-objective (behavioral and genotypic diversity maintenance) and hybrid (Gomes et al, 2015) evolutionary search (Mouret and Doncieux, 2012) and policy transfer (Taylor and Stone, 2009) methods have been separately demonstrated for increasing behavioral quality in various tasks, the impact (on behavior quality) of using non-objective and hybrid evolutionary search in the context of policy transfer remains unknown. Given previous work elucidating the efficacy of hybrid evolutionary search (Gomes and Christensen, 2013b;Gomes et al, 2013Gomes et al, , 2015 and policy transfer (Taylor et al, 2006;Verbancsics and Stanley, 2010;Didi and Nitschke, 2016b) in collective behavior tasks the following hypothesis forms the research focus of this study.…”

“…Given previous work elucidating the efficacy of hybrid evolutionary search (Gomes and Christensen, 2013b;Gomes et al, 2013Gomes et al, , 2015 and policy transfer (Taylor et al, 2006;Verbancsics and Stanley, 2010;Didi and Nitschke, 2016b) in collective behavior tasks the following hypothesis forms the research focus of this study.…”