Improving Hearthstone AI by Combining MCTS and Supervised Learning Algorithms

Abstract

Warning: this is not the final camera-ready version. Abstract-We investigate the impact of supervised prediction models on the strength and efficiency of artificial agents that use the Monte-Carlo Tree Search (MCTS) algorithm to play a popular video game Hearthstone: Heroes of Warcraft. We overview our custom implementation of the MCTS that is well-suited for games with partially hidden information and random effects. We also describe experiments which we designed to quantify the performance of our Hearthstone agent's decision making. We show that even simple neural networks can be trained and successfully used for the evaluation of game states. Moreover, we demonstrate that by providing a guidance to the game state search heuristic, it is possible to substantially improve the win rate, and at the same time reduce the required computations.

Introducing LogDL - Log Description Language for Insights from Complex Data

“…LogDL was in part inspired by our experience with the Grail library aimed at developing AI in video games [24]. Grail supports algorithms such as Utility AI and Monte Carlo Tree Search (MCTS) [25] which can be used as action-selection mechanism for AI players. Utility AI is based on curves that define relationship between an action's utility and a given consideration.…”

The Many AI Challenges of Hearthstone

“…Most of the published academic work on Hearthstone to date focuses on methods for playing the game [30,44,50,53,61]; in addition, there are a few papers about the closely related challenge of playing Magic [57]. Also, the several open-source simulators of Hearthstone mentioned previously are packaged with their own gameplaying agents.…”

“…Also, the several open-source simulators of Hearthstone mentioned previously are packaged with their own gameplaying agents. Most of the published work builds on Monte Carlo Tree Search (MCTS), a stochastic forward planning algorithm initially developed for Go but which has since seen much wider usage, and seeks to find ways the algorithm can be made to work with the game [44,53,62]. A key problem for tree search approaches is how to deal with that the agent does not know the opponent's hand.…”

Survey of Artificial Intelligence for Card Games and Its Application to the Swiss Game Jass

“…Schäfer [34] used UCT to build an AI for Skat, which is still sub-human but comparable to the MC Simulation based player proposed by Kupferschmid et al [29]. Swiechowski et al [35] combined an MCTS player with supervised learning on the logs of sample games, achieving par-human performance. Santos et al [36] outperformed basic MCTS based AIs by combining it with domain-specific knowledge.…”