Optimizing Hearthstone agents using an evolutionary algorithm

García‐Sánchez, Pablo; Tonda, Alberto; Fernández-Leiva, Antonio J.; Cotta, Carlos

doi:10.1016/j.knosys.2019.105032

Cited by 25 publications

(11 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hearthstone [18] is a two-player, turn-taking adversarial online collectable card game that is an increasingly popular domain for evaluating both classical AI techniques and modern deep reinforcement learning approaches due to the many unique challenges it poses (e.g., large branching factor, partial observability, stochastic actions, and difficulty with planning under uncertainty) [33]. Rather than manipulating the reward function of individual agents in a QD system [2,43] (like the QD approach in AlphaStar [53]), or generate the best gameplay strategy or deck [5,25,50], and deckbuilding work of Fontaine et al [22], the experiments in this paper search for a diversity of gameplay policies.…”

Section: Hearthstonementioning

confidence: 99%

Covariance matrix adaptation for the rapid illumination of behavior space

Fontaine

Togelius

Nikolaidis

et al. 2020

Proceedings of the 2020 Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

Quality Diversity (QD) algorithms like Novelty Search with Local Competition (NSLC) and MAP-Elites are a new class of populationbased stochastic algorithms designed to generate a diverse collection of quality solutions. Meanwhile, variants of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) are among the best-performing derivative-free optimizers in single-objective continuous domains. This paper proposes a new QD algorithm called Covariance Matrix Adaptation MAP-Elites (CMA-ME). Our new algorithm combines the dynamic self-adaptation techniques of CMA-ES with archiving and mapping techniques for maintaining diversity in QD.Results from experiments with standard continuous optimization benchmarks show that CMA-ME finds betterquality solutions than MAP-Elites; similarly, results on the strategic game Hearthstone show that CMA-ME finds both a higher overall quality and broader diversity of strategies than both CMA-ES and MAP-Elites. Overall, CMA-ME more than doubles the performance of MAP-Elites using standard QD performance metrics. These results suggest that QD algorithms augmented by operators from state-of-the-art optimization algorithms can yield high-performing methods for simultaneously exploring and optimizing continuous search spaces, with significant applications to design, testing, and reinforcement learning among other domains. Code is available for both the continuous optimization benchmark [21] and Hearthstone [20] domains.

show abstract

Section: Hearthstonementioning

confidence: 99%

Covariance matrix adaptation for the rapid illumination of behavior space

Fontaine

Togelius

Nikolaidis

et al. 2020

Proceedings of the 2020 Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

show abstract

“…Due to their distinct merits of strong robustness, wide applicability and rapid searching capability, the EAs have shown tremendous potential in dealing with the global multiagent optimization problems in practical applications [21]. For example, in [6], the task deadlock problem (caused by agent decision) has been avoided through dynamic and variable pheromone placement methods.…”

Section: Related Workmentioning

confidence: 99%

Task Allocation on Layered Multiagent Systems: When Evolutionary Many-Objective Optimization Meets Deep Q-Learning

Wang

et al. 2021

IEEE Trans. Evol. Computat.

View full text Add to dashboard Cite

This paper is concerned with the multi-task multiagent allocation problem via many-objective optimization for multi-agent systems (MASs). First, a novel layered MAS model is constructed to address the multi-task multi-agent allocation problem that includes both the original task simplification and the many-objective allocation. In the first layer of the model, the deep Q-learning method is introduced to simplify the prioritization of the original task set. In the second layer of the model, the modified shift-based density estimation (MSDE) method is put forward to improve the conventional Strength Pareto Evolutionary Algorithm 2 (SPEA2) in order to achieve many-objective optimization on task assignments. Then, an MSDE-SPEA2-based method is proposed to tackle the many-objective optimization problem with objectives including task allocation, makespan, agent satisfaction, resource utilization, task completion, and task waiting time. As compared with existing allocation methods, the developed method in this paper exhibits an outstanding feature that the task assignment and the task scheduling are carried out simultaneously. Finally, extensive experiments are conducted to 1) verify the validity of the proposed model and the effectiveness of two main algorithms; and 2) illustrate the optimal solution for task allocation and efficient strategy for task scheduling under different scenarios.

show abstract

“…In addition to deckbuilding, Hearthstone presents a unique agentbased challenge due to the stochasticity of initial state and actions, the large branching factor, the partial observability of the game state, and the large variety of opponents possible. As a result, many works train AI agents to play Hearthstone [16][17][18][19][20][21][22][23][24][25]. Other works predict the result of a game given a partial log of games [26,27] or predict the archetype of a deck from the first round of the game [28].…”

Section: Background 21 Hearthstone and Automated Deckbuildingmentioning

confidence: 99%

Deep Surrogate Assisted MAP-Elites for Automated Hearthstone Deckbuilding

Zhang¹,

Fontaine²,

Hoover³

et al. 2021

Preprint

View full text Add to dashboard Cite

We study the problem of efficiently generating high-quality and diverse content in games. Previous work on automated deckbuilding in Hearthstone shows that the quality diversity algorithm MAP-Elites can generate a collection of high-performing decks with diverse strategic gameplay. However, MAP-Elites requires a large number of expensive evaluations to discover the diverse collection of decks. We propose assisting MAP-Elites with a deep surrogate model trained online to predict game outcomes with respect to candidate decks. MAP-Elites discovers a diverse dataset to improve the surrogate model accuracy, while the surrogate model helps guide MAP-Elites towards promising new content. In a Hearthstone deckbuilding case study, we show that our approach improves the sample efficiency of MAP-Elites and outperforms a model trained offline with random decks, as well as a linear surrogate model baseline, setting a new state-of-the-art for quality diversity approaches in the application domain of automated Hearthstone deckbuilding.

show abstract

Optimizing Hearthstone agents using an evolutionary algorithm

Cited by 25 publications

References 13 publications

Covariance matrix adaptation for the rapid illumination of behavior space

Covariance matrix adaptation for the rapid illumination of behavior space

Task Allocation on Layered Multiagent Systems: When Evolutionary Many-Objective Optimization Meets Deep Q-Learning

Deep Surrogate Assisted MAP-Elites for Automated Hearthstone Deckbuilding

Contact Info

Product

Resources

About