Modifications of UCT and sequence-like simulations for Monte-Carlo Go

Abstract

Abstract-Algorithm UCB1 for multi-armed bandit problem has already been extended to Algorithm UCT which works for minimax tree search. We have developed a Monte-Carlo program, MoGo, which is the first computer Go program using UCT. We explain our modification of UCT for Go application and also the sequence-like random simulation with patterns which has improved significantly the performance of MoGo. UCT combined with pruning techniques for large Go board is discussed, as well as parallelization of UCT. MoGo is now a top-level Computer-Go program on 9 × 9 Go board.

Adding Expert Knowledge and Exploration in Monte-Carlo Tree Search

“…Second, reducing diversity has been a good idea in Monte-Carlo; [13] has shown that introducing several patterns and rule greatly improve the efficiency of Monte-Carlo Tree-Search. However, plenty of experiments around increasing the level of the Monte-Carlo simulator as a stand-alone player have given negative results -diversity and playing strength are too conflicting objectives.…”

“…Many people have tried to improve the MC engine by increasing its level (the strength of the Monte-Carlo simulator as a standalone player), but it is shown clearly in [13,10] that this is not the good criterion: a MC engine M C 1 which plays significantly better than another M C 2 can lead to very poor results as a module in MCTS, whenever the computational cost is the same. Some MC engines have been learnt on datasets [8], but the results are strongly improved by changing the constants manually.…”

“…In that sense, designing and calibrating a MC engine remains an open challenge: one has to intensively experiment a modification in order to validate it. Various shapes are defined in [2,13,12]. [13] uses patterns and expertise as explained in Algorithm 1.…”

“…Various shapes are defined in [2,13,12]. [13] uses patterns and expertise as explained in Algorithm 1. We present below two new improvements, both of them centered on an increased diversity when the computational power increases; in both cases, the improvement is negative or negligible for small computational power and becomes highly significant when the computational power increases.…”

Learning a Move-Generator for Upper Confidence Trees

“…It has been greatly improved by including Progressive Widening and Double Progressive Widening [6,2], RAVE values [7], Blind Values [4], and handcrafted Monte-Carlo moves [17,10]. A crucial component is the Monte-Carlo move generator, also known as the playout generator.…”