A Bayesian model for RTS units control applied to StarCraft

Abstract

The task of keyhole (unobtrusive) plan recognition is central to adaptive game AI. "Tech trees" or "build trees" are the core of real-time strategy (RTS) game strategic (long term) planning. This paper presents a generic and simple Bayesian model for RTS build tree prediction from noisy observations, which parameters are learned from replays (game logs). This unsupervised machine learning approach involves minimal work for the game developers as it leverage players' data (common in RTS). We applied it to StarCraft 1 and showed that it yields high quality and robust predictions, that can feed an adaptive AI.

A Review of Real‐Time Strategy Game AI

“…StarCraft is a canonical RTS game, like chess is to board games, with a huge player base and numerous professional competitions. The game has three different but very well balanced teams, or races, allowing for varied strategies and tactics without any dominant strategy, and requires both strategic and tactical decision making roughly equally (Synnaeve and Bessière 2011b). These features give StarCraft an advantage over other RTS titles that are used for AI research, such as Wargus 2 and ORTS.…”

“…They have a vast space of actions and game states, with durative actions, a huge branching factor, and actions that can have longterm effects throughout the course of a match Buro and Furtak 2004;Mehta et al 2009;Ontañón 2012;Tozour 2002;. Even compared with Go, which is currently an active area of AI research, RTS games present a huge increase in complexity -at least an order of magnitude increase in the number of possible game states, actions to choose from, actions per game, and actions per minute (using standard rules) (Buro 2004;Schaeffer 2001;Synnaeve and Bessière 2011b). The state space is so large that traditional heuristic-based search techniques, which have proven effective in a range of board games (Schaeffer 2001), have so far been unable to solve all but the most restricted subproblems of RTS AI.…”

“…StarCraft has a lasting popularity among professional and amateur players, including a large professional gaming scene in South Korea, with international competitions awarding millions of dollars in prizes every year (Churchill and Buro 2011). This popularity means that there are a large number of high-quality game logs (replays) available on the Internet that can be used for data mining, and there are many players of all skill levels to test against Synnaeve and Bessière 2011b;Weber, Mateas, and Jhala 2011a).…”

“…Due to their complexity and challenges, RTS games are probably the best current environment in which to pursue Laird and van Lent's vision of game AI as a stepping stone toward human-level AI. It is a particularly interesting area for AI research because even the best agents are outmatched by experienced humans (Huang 2011;Synnaeve and Bessière 2011a;, due to the human abilities to abstract, reason, learn, plan, and recognize plans (Buro 2004;Buro and Churchill 2012).…”

“…Synnaeve and Bessière (2011b) combine unit objectives, opportunities, and threats using a Bayesian model to decide which direction to move units in a battle. The model treats each of its sensory inputs as part of a probability equation that can be solved, given data (potentially learned through RL) about the distributions of the inputs with respect to the direction moved, to find the probability that a unit should move in each possible direction.…”