Multi-Emitter MAP-Elites: Improving quality, diversity and convergence speed with heterogeneous sets of emitters

Cully, Antoine

doi:10.48550/arxiv.2007.05352

Cited by 5 publications

(10 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, it may be possible to automatically and dynamically adapt the evaluation budget of the collection of containers by using a multi-container version of ME-MAP-Elites [12] to dynamically give more evaluation budget to containers depending on their statistics such as coverage, QD-score, redundancy, best-ever, etc. This topic will be addressed in a future paper.…”

Section: Discussionmentioning

confidence: 99%

Ensemble Feature Extraction for Multi-Container Quality-Diversity Algorithms

Cazenille

2021

Preprint

View full text Add to dashboard Cite

Quality-Diversity algorithms search for large collections of diverse and high-performing solutions, rather than just for a single solution like typical optimisation methods. They are specially adapted for multi-modal problems that can be solved in many different ways, such as complex reinforcement learning or robotics tasks. However, these approaches are highly dependent on the choice of feature descriptors (FDs) quantifying the similarity in behaviour of the solutions. While FDs usually needs to be hand-designed, recent studies have proposed ways to define them automatically by using feature extraction techniques, such as PCA or Auto-Encoders, to learn a representation of the problem from previously explored solutions.Here, we extend these approaches to more complex problems which cannot be efficiently explored by relying only on a single representation but require instead a set of diverse and complementary representations. We describe MC-AURORA, a Quality-Diversity approach that optimises simultaneously several collections of solutions, each with a different set of FDs, which are, in turn, defined automatically by an ensemble of modular auto-encoders. We show that this approach produces solutions that are more diverse than those produced by single-representation approaches.

show abstract

Section: Discussionmentioning

confidence: 99%

Ensemble Feature Extraction for Multi-Container Quality-Diversity Algorithms

Cazenille

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…An alternative method that has been shown to increase the sample efficiency of QD is the addition of more complex optimization techniques. When combined with policy gradients [35] and evolution strategies [26], [36], [37], the efficiency of QD algorithms can be improved. Our work is complementary to this line of work and these methods can easily be incorporated into our framework in future work.…”

Section: Related Workmentioning

confidence: 99%

Dynamics-Aware Quality-Diversity for Efficient Learning of Skill Repertoires

Lim,

Grillotti,

Bernasconi

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Quality-Diversity (QD) algorithms are powerful exploration algorithms that allow robots to discover large repertoires of diverse and high-performing skills. However, QD algorithms are sample inefficient and require millions of evaluations. In this paper, we propose Dynamics-Aware Quality-Diversity (DA-QD), a framework to improve the sample efficiency of QD algorithms through the use of dynamics models. We also show how DA-QD can then be used for continual acquisition of new skill repertoires. To do so, we incrementally train a deep dynamics model from experience obtained when performing skill discovery using QD. We can then perform QD exploration in imagination with an imagined skill repertoire. We evaluate our approach on three robotic experiments. First, our experiments show DA-QD is 20 times more sample efficient than existing QD approaches for skill discovery. Second, we demonstrate learning an entirely new skill repertoire in imagination to perform zero-shot learning. Finally, we show how DA-QD is useful and effective for solving a long horizon navigation task and for damage adaptation in the real world. Videos and source code are available at: https://sites.google.com/view/da-qd.

show abstract

“…QD algorithms have been extended by combining them with ES [3] to increase their efficiency and speed of convergence [9,10,18]. In [9], an ES uses NS's novelty objective to look for novel solutions while improving their performances.…”

Section: Divergent Search Algorithmsmentioning

confidence: 99%

“…In [9], an ES uses NS's novelty objective to look for novel solutions while improving their performances. At the same time, the approach followed in [18], and then extended in [10], uses ME as a scheduler for modified instances of CMA-ES [22], named emitters. Exploration of the search space and reward exploitation are both performed through emitters.…”

Section: Divergent Search Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation

Sparse reward exploration via novelty search and emitters

Paolo

Coninx

Doncieux

et al. 2021

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

In figure 7 we show the distribution of the behaviors of the policies in the final archive. Each point is represents different policy. In blue are the policies that do not get any reward, thus considered exploratory, while in orange are rewarding policies, considered exploitative. For SERENE the exploratory policies are the ones in the novelty archive A 𝑁 , while the exploitative policies are the ones in the rewarding archive A 𝑅 .We can see that even if the coverage metric values for SERENE are lower with respect to ME, the search space is well covered. Moreover, the reward areas are densely explored.

show abstract

Multi-Emitter MAP-Elites: Improving quality, diversity and convergence speed with heterogeneous sets of emitters

Cited by 5 publications

References 29 publications

Ensemble Feature Extraction for Multi-Container Quality-Diversity Algorithms

Ensemble Feature Extraction for Multi-Container Quality-Diversity Algorithms

Dynamics-Aware Quality-Diversity for Efficient Learning of Skill Repertoires

Sparse reward exploration via novelty search and emitters

Contact Info

Product

Resources

About