Discovering representations for black-box optimization

Gaier, Adam; Asteroth, Alexander; Mouret, Jean-Baptiste

doi:10.1145/3377930.3390221

Cited by 37 publications

(26 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the focus was on the application of UCB for parent selection, this paper has also made contributions on the use of offspring survival as a reward mechanism. As noted in Section 2, offspring survival has been identified as an important measure [3,11] and as a milestone in an individual's lifetime [6]. This paper contributes to earlier work by exploring how survival is assessed (based on the individual or the cell).…”

Section: Discussionmentioning

confidence: 88%

“…Moreover, the UCB formula in [10] takes into account performance and variance of performance, rather than traces of the evolutionary progress as in this paper. In other work [11], UCB1 is applied to choose which of the genetic operators (re-constructive crossover, line mutation, or isometric mutation) to apply to the parent. Similar to this paper, the reward in [11] "is assigned in proportion to the number of children who earned a place in the archive".…”

Section: Introductionmentioning

confidence: 99%

“…In other work [11], UCB1 is applied to choose which of the genetic operators (re-constructive crossover, line mutation, or isometric mutation) to apply to the parent. Similar to this paper, the reward in [11] "is assigned in proportion to the number of children who earned a place in the archive".…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Monte Carlo elites

Sfikas

Liapis

Yannakakis

2021

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

A core challenge of evolutionary search is the need to balance between exploration of the search space and exploitation of highly fit regions. Quality-diversity search has explicitly walked this tightrope between a population's diversity and its quality. This paper extends a popular quality-diversity search algorithm, MAP-Elites, by treating the selection of parents as a multi-armed bandit problem. Using variations of the upper-confidence bound to select parents from under-explored but potentially rewarding areas of the search space can accelerate the discovery of new regions as well as improve its archive's total quality. The paper tests an indirect measure of quality for parent selection: the survival rate of a parent's offspring. Results show that maintaining a balance between exploration and exploitation leads to the most diverse and high-quality set of solutions in three different testbeds. CCS CONCEPTS• Computing methodologies → Continuous space search; • Applied computing → Computer games.

show abstract

Section: Discussionmentioning

confidence: 88%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Monte Carlo elites

Sfikas

Liapis

Yannakakis

2021

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

show abstract

“…In order to answer Q1, we compare the performance of the proposed PoMS approach, to MAP-Elites (MAPE-Iso) which performs search in the original parameter space. Within this analysis, we also consider two state-of-the-art baselines which are based on a notion of a manifold: MAP-Elites via Elite Hypervolumes (MAPE-IsoLineDD) [46] and Data-driven encoding MAP-Elites (DDE) [18].…”

Section: Discussionmentioning

confidence: 99%

“…DDE hyperparameters consist of the AE architecture and mutation operator specific hyperparameters. The former is kept the same as with PoMS, for each of the experiments, while the latter are as in the original paper [18]. Instead of running a fixed window for the multi-armed bandit upper confidence bound operator selector, we maintain a moving average.…”

Section: B Implementation Detailsmentioning

confidence: 99%

Policy manifold search

Rakicevic

Cully

Kormushev

2021

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

Neuroevolution is an alternative to gradient-based optimisation that has the potential to avoid local minima and allows parallelisation. The main limiting factor is that usually it does not scale well with parameter space dimensionality. Inspired by recent work examining neural network intrinsic dimension and loss landscapes, we hypothesise that there exists a low-dimensional manifold, embedded in the policy network parameter space, around which a high-density of diverse and useful policies are located. This paper proposes a novel method for diversity-based policy search via Neuroevolution, that leverages learned representations of the policy network parameters, by performing policy search in this learned representation space. Our method relies on the Quality-Diversity (QD) framework which provides a principled approach to policy search, and maintains a collection of diverse policies, used as a dataset for learning policy representations. Further, we use the Jacobian of the inversemapping function to guide the search in the representation space. This ensures that the generated samples remain in the high-density regions, after mapping back to the original space. Finally, we evaluate our contributions on four continuous-control tasks in simulated environments, and compare to diversity-based baselines. CCS CONCEPTS• Computing methodologies → Dimensionality reduction and manifold learning; Evolutionary robotics; Neural networks.

show abstract

Quality-Diversity Optimization: A Novel Branch of Stochastic Optimization

Chatzilygeroudis

Cully

Vassiliades

et al. 2021

Springer Optimization and Its Applications

Self Cite

View full text Add to dashboard Cite

Traditional optimization algorithms search for a single global optimum that maximizes (or minimizes) the objective function. Multimodal optimization algorithms search for the highest peaks in the search space that can be more than one. Quality-Diversity algorithms are a recent addition to the evolutionary computation toolbox that do not only search for a single set of local optima, but instead try to illuminate the search space. In effect, they provide a holistic view of how high-performing solutions are distributed throughout a search space. The main differences with multimodal optimization algorithms are that (1) Quality-Diversity typically works in the behavioral space (or feature space), and not in the genotypic (or parameter) space, and (2) Quality-Diversity attempts to fill the whole behavior space, even if the niche is not a peak in the fitness landscape. In this chapter, we provide a gentle introduction to Quality-Diversity optimization, discuss the main representative algorithms, and the main current topics under consideration in the community. Throughout the chapter, we also discuss several successful applications of Quality-Diversity algorithms, including deep learning, robotics, and reinforcement learning.

show abstract

Discovering representations for black-box optimization

Cited by 37 publications

References 38 publications

Monte Carlo elites

Monte Carlo elites

Policy manifold search

Quality-Diversity Optimization: A Novel Branch of Stochastic Optimization

Contact Info

Product

Resources

About