Monte Carlo Tree Search with heuristic evaluations using implicit minimax backups

Lanctot, Marc; Winands, Mark H. M.; Pepels, Tom; Sturtevant, Nathan R.

doi:10.1109/cig.2014.6932903

Cited by 27 publications

(15 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…by Ramanujan and Selman (2011), where the algorithm U CT M AX H replaces MCTS rollouts with heuristic evaluations and classic averaging MCTS backups with minimaxing backups. In implicit minimax backups (Lanctot et al, 2014), both minimaxing backups of heuristic evaluations and averaging backups of rollout returns are managed simultaneously. On the other hand, one can nest minimax searches into MCTS searches.…”

Section: Related Workmentioning

confidence: 99%

MCTS-Minimax Hybrids with State Evaluations

Baier¹,

Winands²

2018

jair

Self Cite

View full text Add to dashboard Cite

Monte-Carlo Tree Search (MCTS) has been found to show weaker play than minimaxbased search in some tactical game domains. This is partly due to its highly selective search and averaging value backups, which make it susceptible to traps. In order to combine the strategic strength of MCTS and the tactical strength of minimax, MCTS-minimax hybrids have been introduced, embedding shallow minimax searches into the MCTS framework. Their results have been promising even without making use of domain knowledge such as heuristic evaluation functions. This article continues this line of research for the case where evaluation functions are available. Three different approaches are considered, employing minimax with an evaluation function in the rollout phase of MCTS, as a replacement for the rollout phase, and as a node prior to bias move selection. The latter two approaches are newly proposed. Furthermore, all three hybrids are enhanced with the help of move ordering and k-best pruning for minimax. Results show that the use of enhanced minimax for computing node priors results in the strongest MCTS-minimax hybrid investigated in the three test domains of Othello, Breakthrough, and Catch the Lion. This hybrid, called MCTS-IP-M-k, also outperforms enhanced minimax as a standalone player in Breakthrough, demonstrating that at least in this domain, MCTS and minimax can be combined to an algorithm stronger than its parts. Using enhanced minimax for computing node priors is therefore a promising new technique for integrating domain knowledge into an MCTS framework.

show abstract

Section: Related Workmentioning

confidence: 99%

MCTS-Minimax Hybrids with State Evaluations

Baier¹,

Winands²

2018

jair

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [29], the researchers proposed implicit minimax backups which incorporated into MCTS the minimax scores based on heuristic evaluations of positions. The minimax scores were used together with the estimator Q i to guide the selections of MCTS.…”

Section: Previous Workmentioning

confidence: 99%

“…One issue to discuss is the minimax scores for chance nodes, which were not mentioned in [29]. An intuitive way is to use the probability distribution for unrevealed pieces, when calculating the expected values for chance nodes.…”

Section: Our Workmentioning

confidence: 99%

An analysis for strength improvement of an MCTS-based program playing Chinese dark chess

Hsueh

Tseng

et al. 2016

Theoretical Computer Science

View full text Add to dashboard Cite

“…Similar techniques have been shown to significantly improve [24] the quality of MCTS players in games like Breakout and LOA [35]. Lanctot et al [24], introduced adding evaluation function values to nodes in the MCTS tree. These values are updated during the backup phase, by considering the minimax values of a node's children.…”

Section: Related Workmentioning

confidence: 99%

“…Various studies have considered approaches to dealing with stochasticity and uncertain information in MCTS [5,3,31,8,24].…”

Section: Mcts For Stochastic Domainsmentioning

confidence: 99%

Sample Evaluation for Action Selection in Monte Carlo Tree Search

Brand

Kroon

2014

Proceedings of the Southern African Institute for Computer Scientist and Information Technologists Annual Conference 2014 on SA

View full text Add to dashboard Cite

Building sophisticated computer players for games has been of interest since the advent of artificial intelligence research. Monte Carlo tree search (MCTS) techniques have led to recent advances in the performance of computer players in a variety of games. Without any refinements, the commonlyused upper confidence bounds applied to trees (UCT) selection policy for MCTS performs poorly on games with high branching factors, because an inordinate amount of time is spent performing simulations from each sibling of a node before that node can be further investigated. Move-ordering heuristics are usually proposed to address this issue, but when the branching factor is large, it can be costly to order candidate actions. We propose a technique combining sampling from the action space with a naïve evaluation function for identifying nodes to add to the tree when using MCTS in cases where the branching factor is large. The approach is evaluated on a restricted version of the board game Risk with promising results.

show abstract

Monte Carlo Tree Search with heuristic evaluations using implicit minimax backups

Cited by 27 publications

References 28 publications

MCTS-Minimax Hybrids with State Evaluations

MCTS-Minimax Hybrids with State Evaluations

An analysis for strength improvement of an MCTS-based program playing Chinese dark chess

Sample Evaluation for Action Selection in Monte Carlo Tree Search

Contact Info

Product

Resources

About