Helping AI to Play Hearthstone: AAIA’17 Data Mining Challenge

Abstract

Abstract-This paper summarizes the AAIA'17 Data Mining Challenge: Helping AI to Play Hearthstone which was held between March 23, and May 15, 2017 at the Knowledge Pit platform. We briefly describe the scope and background of this competition in the context of a more general project related to the development of an AI engine for video games, called Grail. We also discuss the outcomes of this challenge and demonstrate how predictive models for the assessment of player's winning chances can be utilized in a construction of an intelligent agent for playing Hearthstone. Finally, we show a few selected machine learning approaches for modeling state and action values in Hearthstone. We provide evaluation for a few promising solutions that may be used to create more advanced types of agents, especially in conjunction with Monte Carlo Tree Search algorithms.

The Many AI Challenges of Hearthstone

“…Most of the published academic work on Hearthstone to date focuses on methods for playing the game [30,44,50,53,61]; in addition, there are a few papers about the closely related challenge of playing Magic [57]. Also, the several open-source simulators of Hearthstone mentioned previously are packaged with their own gameplaying agents.…”

“…This missing information makes it impossible to expand the search tree based on the opponent's move to do a minimax search, unless a good guess of what their hand might be is available. Some of the work has therefore focused on learning predictive models of the opponent's hand [15,30]. Other agents, such as that which is part of MetaStone, simply searches up until the end of the current move and uses a heuristic evaluation function, not even attempting to predict the opponent's move.…”

Ensembling a Linear Eegression model with an error Mitigation Component

“…However, thanks to being limited to subset of input variables and being focused on another dimension of the modelling goal (difference between training sample instead of absolute value) there are cases when it performs well. Model ensembling has proven to be an effective way of combining multiple models for the sake of increasing the output accuracy over the single-technique models [2] [3]. However, a typical use case is to combine multiple models made with different techniques and then judge which output is the best or aggregate all the outputs into single model response, for example by taking average, weighted average or sum of multiple components.…”

A Practical Solution to Handling Randomness and Imperfect Information in Monte Carlo Tree Search

“…We describe a relatively complete approach to using MCTS in a game with hidden information and random effects [19], [20]. However, the specific game we have chosen for this study is Hearthstone: Heroes of Warcraft, developed by Blizzard Entertainment [21].…”