An Analysis of Hall-of-Fame Strategies in Competitive Coevolutionary Algorithms for Self-Learning in RTS Games

Nogueira, M. A.; Cotta, Carlos; Fernández-Leiva, Antonio J.

doi:10.1007/978-3-642-44973-4_19

Cited by 11 publications

(20 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, although the contribution of the ''archive methods'' to reduce the number of drawbacks (e.g., missing features and cycling) of coevolutionary systems is undeniable, their implementation entails a number of issues that have to be taken into account such as the strength of the archive members and the control of the archive diversity. So for instance, a high risk of generating cycles can appear if the strength of a member is valued by pitting this member against a fixed set of opponents, and this is precisely what we did in our previous work (Nogueira et al 2012(Nogueira et al , 2013. A key point to consider is the metric that defines the strength of an individual, which should indicate when an individual is qualitatively superior to the rest but without losing sight of the fact that sometimes quantitative superiority does not correspond with a high strength.…”

Section: (Competitive) Coevolution In Gamessupporting

confidence: 67%

“…Using Planet Wars we propose, in this paper, a modified and improved version of a CC algorithm that we already described in Nogueira et al (2012Nogueira et al ( , 2013 for a simplistic capture-the-flag game. The latter used the HoF as a longterm memory mechanism to keep the winning strategies found in each coevolutionary step and all of them were also used in the evaluation process (in the basic algorithm).…”

Section: Hofcc Algorithm and Variantsmentioning

confidence: 99%

“…The work presented here, builds on our previous work (Nogueira et al 2012(Nogueira et al , 2013 on testing the use of HoFbased CC (HoFCC) algorithms for finding winning strategies in RTS games. In our previous papers we analyzed how the diversity and the growth of the HoF can influence the quality of the solutions obtained by HoFCC algorithms.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Virtual player design using self-learning via competitive coevolutionary algorithms

2014

Self Cite

View full text Add to dashboard Cite

The Google Artificial Intelligence (AI) Challenge is an international contest the objective of which is to program the AI in a two-player real time strategy (RTS) game. This AI is an autonomous computer program that governs the actions that one of the two players executes during the game according to the state of play. The entries are evaluated via a competition mechanism consisting of two-player rounds where each entry is tested against others. This paper describes the use of competitive coevolutionary (CC) algorithms for the automatic generation of winning game strategies in Planet Wars, the RTS game associated with the 2010 contest. Three different versions of a prime algorithm have been tested. Their common nexus is not only the use of a Hall-of-Fame (HoF) to keep note of the winners of past coevolutions but also the employment of an archive of experienced players, termed the hall-of-celebrities (HoC), that puts pressure on the optimization process and guides the search to increase the strength of the solutions; their differences come from the periodical updating of the HoF on the basis of quality and diversity metrics. The goal is to optimize the AI by means of a self-learning process guided by coevolutionary search and competitive evaluation. An empirical study on the performance of a number of variants of the proposed algorithms is described and a statistical analysis of the results is conducted. In addition to the attainment of competitive bots we also conclude that the incorporation of the HoC inside the primary algorithm helps to reduce the effects of cycling caused by the use of HoF in CC algorithms.

show abstract

Section: (Competitive) Coevolution In Gamessupporting

confidence: 67%

Section: Hofcc Algorithm and Variantsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Virtual player design using self-learning via competitive coevolutionary algorithms

2014

Self Cite

View full text Add to dashboard Cite

show abstract

“…-G decides which policies in the menagerie, π ∈ π o , will be discarded from the menagerie. The curator bears resemblance with the notion of Hall of Fame from evolutionary algorithms [23]. As Hall of Fame algorithms also consider the problem of curating a policy set over time.…”

Section: A Framework Definitionmentioning

confidence: 99%

A Comparison of Self-Play Algorithms Under a Generalized Framework

et al. 2022

View full text Add to dashboard Cite

The notion of self-play, albeit often cited in multiagent Reinforcement Learning as a process by which to train agent policies from scratch, has received little efforts to be taxonomized within a formal model. We present a formalized framework, with clearly defined assumptions, which encapsulates the meaning of self-play as abstracted from various existing selfplay algorithms. This framework is framed as an approximation to a theoretical solution concept for multiagent training. Through a novel qualitative visualization metric, on a simple environment, we show that different self-play algorithms generate different distributions of episode trajectories, leading to different explorations of the policy space by the learning agents. Quantitatively, on two environments, we analyze the learning dynamics of policies trained under different self-play algorithms captured under our framework and perform cross self-play performance comparisons. Our results indicate that, throughout training, various widely used self-play algorithms exhibit cyclic policy evolutions and that the choice of self-play algorithm significantly affects the final performance of trained agents.

show abstract

“…Even though the archive mechanisms such as the Hall of Fame (HoF) [9,6,5] have been studied in the past and are known to maintain progress in an evolutionary arms race, they do not provide any specific guarantees in terms of convergence to the optimal solution. Moreover, the characteristics of archive's influence on evolving individuals is little known, which constitutes the main motivation for this study.…”

Section: Introductionmentioning

confidence: 99%

Quantitative analysis of the hall of fame coevolutionary archives

Liskowski

2013

Proceedings of the 15th Annual Conference Companion on Genetic and Evolutionary Computation

View full text Add to dashboard Cite

This paper provides an attempt to investigate the properties of the Hall of Fame archive in two-population competitive coevolution environment applied to the game of Othello. Using the measure of expected utility, a round-robin tournament and performance profiles, we show that coevolution can be biased towards playing better with stronger opponents if it is driven by interactions with the past champions kept in the archive, in addition to pure competition among coevolving individuals. Moreover, the Hall of Fame does not necessarily influence the overall perfromance in terms of expected utility, as it trades-off the ability to cope with opponents of various strength, so that the resulting players are more likely to win with a strong opponent than with a weak one.

show abstract

An Analysis of Hall-of-Fame Strategies in Competitive Coevolutionary Algorithms for Self-Learning in RTS Games

Cited by 11 publications

References 8 publications

Virtual player design using self-learning via competitive coevolutionary algorithms

Virtual player design using self-learning via competitive coevolutionary algorithms

A Comparison of Self-Play Algorithms Under a Generalized Framework

Quantitative analysis of the hall of fame coevolutionary archives

Contact Info

Product

Resources

About