Scaling Monte Carlo Tree Search on Intel Xeon Phi

Abstract

Abstract-Many algorithms have been parallelized successfully on the Intel Xeon Phi coprocessor, especially those with regular, balanced, and predictable data access patterns and instruction flows. Irregular and unbalanced algorithms are harder to parallelize efficiently. They are, for instance, present in artificial intelligence search algorithms such as Monte Carlo Tree Search (MCTS). In this paper we study the scaling behavior of MCTS, on a highly optimized realworld application, on real hardware. The Intel Xeon Phi allows shared memory scaling studies up to 61 cores and 244 hardware threads. We compare work-stealing (Cilk Plus and TBB) and work-sharing (FIFO scheduling) approaches. Interestingly, we find that a straightforward thread pool with a work-sharing FIFO queue shows the best performance. A crucial element for this high performance is the controlling of the grain size, an approach that we call Grain Size Controlled Parallel MCTS. Our subsequent comparing with the Xeon CPUs shows an even more comprehensible distinction in performance between different threading libraries. We achieve, to the best of our knowledge, the fastest implementation of a parallel MCTS on the 61 core Intel Xeon Phi using a real application (47 relative to a sequential run).

Monte Carlo Tree Search: a review of recent modifications and applications

“…Secondly, the programmer has no control over thread scheduling and CPU/ GPU communication, which can make the approach risky to apply in tournament scenarios that require precise time controls. Mirsoleimani et al (2015) study the MCTS method running in parallel on Intel Xeon Phi, which is a processing unit designed for parallel computations. Each unit allows shared memory scaling up to 61 cores and 244 threads.…”

The HEPGAME project

“…For this one of the computers we bought was equiped with an Intel Phi board. He spent a lot of time studying this and made some very nice measurements of its performance which were presented at conferences (the major way of presenting results in this field) [8,9], but the eventual conclusion was that unfortunately it was not going to help in physics calculations based on computer algebra. I am not talking about for instance Monte Carlo integration and that type of calculations, but only about symbolic manipulations.…”

A New Method for Parallel Monte Carlo Tree Search

“…Among the available methods, tree parallelization is one of the most suitable techniques for shared-memory platforms [1]. Good scalability in tree parallelization has two distinct flavors [5], [6]:…”

“…Generally, implementing tree parallelization for MCTS often creates a dilemma. We can either use a small granularity algorithm with a high playout-speedup but also low strength-speedup, or we can use a larger granularity algorithm that has high strength-speedup but with a lower playout-speedup [6], [7]. A version of the parallel MCTS that is both playout-scalable and strength-scalable , has the best parallel efficiency.…”