Monte Carlo Tree Search in Lines of Action

Abstract

Abstract-The success of Monte-Carlo Tree Search (MCTS) in many games where αβ-search has failed, naturally raises the question whether Monte-Carlo simulations will eventually also outperform traditional game-tree search in game domains where αβ-based search is now successful. The forte of αβ-search are highly-tactical deterministic game domains with a small to moderate branching factor, where efficient yet knowledge-rich evaluation functions can be applied effectively.In here we describe a MCTS-based program for playing the game Lines of Action (LOA), which is a highly-tactical slowprogression game exhibiting many of the properties difficult for MCTS. The program uses an improved MCTS variant that allows it to both prove the game-theoretical value of nodes in a search tree and to focus its simulations better using domain knowledge. This results in simulations superior in both handling tactics and ensuring game progression. Using the improved MCTS variant, our program is able to outperform even the world's strongest αβ-based LOA program. This is an important milestone for MCTS because the traditional game-tree search approach has been considered to be the better suited for playing LOA.

Sample Evaluation for Action Selection in Monte Carlo Tree Search

“…Similar techniques have been shown to significantly improve [24] the quality of MCTS players in games like Breakout and LOA [35]. Lanctot et al [24], introduced adding evaluation function values to nodes in the MCTS tree.…”

An analysis for strength improvement of an MCTS-based program playing Chinese dark chess

“…In [44,45], their methods for Lines of Action (LOA) were to check the scores by evaluation function every three moves, and then return the results in the following cases. If the scores exceeded some threshold, the results were wins.…”

An Automatically Generated Evaluation Function in General Game Playing

“…Still, in many cases (e.g., Go [7], Amazons [15], Lines of Action [29], and GGP itself [4], [26]), it seems that the UCT can benefit from the introduction of some sort of an additional, possibly game-specific, bias into the simulations. Below, we discuss several possibilities of combining the UCT with our automatically inferred game-specific state evaluation function into what we call GUCT algorithm.…”

“…Below, we discuss several possibilities of combining the UCT with our automatically inferred game-specific state evaluation function into what we call GUCT algorithm. While there are publications (e.g., the aforementioned [29]) that propose alternative solutions, we concentrate here on universal, generally effective approaches that seem especially well suited for the GGP research framework.…”