Monte Carlo Tree Search: a review of recent modifications and applications

Świechowski, Maciej; Godlewski, Konrad; Sawicki, Bartosz; Mańdziuk, Jacek

doi:10.1007/s10462-022-10228-y

Cited by 103 publications

(56 citation statements)

References 206 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MCTS is a powerful global optimization method and is very popular in computer gaming algorithms such as Alpha Go, Bridge, Poker, and many other video games. , It has recently been adopted for materials design problems. ,− It integrates a tree search algorithm with reinforcement learning. , The algorithm begins with building a shallow tree of nodes, where each node represents a point in the search space. It subsequently generates downstream pathways by a rollout procedure.…”

Section: Design Workflowsmentioning

confidence: 99%

Data-Driven Methods for Accelerating Polymer Design

Patra

2021

ACS Polym. Au

View full text Add to dashboard Cite

Optimal design of polymers is a challenging task due to their enormous chemical and configurational space. Recent advances in computations, machine learning, and increasing trends in data and software availability can potentially address this problem and accelerate the molecularscale design of polymers. Here, the central problem of polymer design is reviewed, and the general ideas of data-driven methods and their working principles in the context of polymer design are discussed. This Review provides a historical perspective and a summary of current trends and outlines future scopes of data-driven methods for polymer research. A few representative case studies on the use of such data-driven methods for discovering new polymers with exceptional properties are presented. Moreover, attempts are made to highlight how data-driven strategies aid in establishing new correlations and advancing the fundamental understanding of polymers. This Review posits that the combination of machine learning, rapid computational characterization of polymers, and availability of large open-sourced homogeneous data will transform polymer research and development over the coming decades. It is hoped that this Review will serve as a useful reference to researchers who wish to develop and deploy data-driven methods for polymer research and education.

show abstract

Section: Design Workflowsmentioning

confidence: 99%

Data-Driven Methods for Accelerating Polymer Design

Patra

2021

ACS Polym. Au

View full text Add to dashboard Cite

show abstract

“…The Monte Carlo Tree Search (MCTS) algorithm [31][32][33] is a best-first search algorithm that does not require any prior data and can iteratively gather information to enhance decisionmaking. Although computationally expensive, MCTS has been very popular in the literature [31,34], especially in long-horizon planning applications where no immediate reward is available.…”

Section: Monte Carlo Tree Searchmentioning

confidence: 99%

Learning to Design Without Prior Data: Discovering Generalizable Design Strategies Using Deep Learning and Tree Search

Raina

Cagan

McComb

2022

Journal of Mechanical Design

View full text Add to dashboard Cite

Building an AI agent that can design on its own has been a goal since the 1980s. Recently, deep learning has shown the ability to learn from large-scale data, enabling significant advances in data-driven design. However, learning over prior data limits us only to solve problems that have been solved before and biases us towards existing solutions. The ultimate goal for a design agent is the ability to learn generalizable design behavior in a problem space without having seen it before. We introduce a self-learning agent framework in this work that achieves this goal. This framework integrates a deep policy network with a novel tree search algorithm, where the tree search explores the problem space, and the deep policy network leverages self-generated experience to guide the search further. This framework first demonstrates an ability to discover high-performing generative strategies without any prior data, and second, it illustrates a zero-shot generalization of generative strategies across various unseen boundary conditions. This work evaluates the effectiveness and versatility of the framework by solving multiple versions of two engineering design problems without retraining. Overall, this paper presents a methodology to self-learn high-performing and generalizable problem-solving behavior in an arbitrary problem space, circumventing the needs for expert data, existing solutions, and problem-specific learning.

show abstract

“…The framework of Markov Decision Processes (MDPs) is a useful basis for optimal control, planning, and learning in robots [2], [12]. Classic fully-observable problems have a rich history with a variety of effective solution techniques [1]; recent work has sought extensions to the basic MDP formulation to capture additional features including time-varying models [3] and more complex representations [4], as well as exploring various means to improve performance, especially in solving very large instances [14]. The settings we will consider are not fully-observable (except in the degenerate case with unit period), and it so might be better considered partially observable.…”

Section: B Related Workmentioning

confidence: 99%

Planning under periodic observations: bounds and bounding-based solutions

Rossi¹,

Shell²

2022

Preprint

View full text Add to dashboard Cite

We study planning problems faced by robots operating in uncertain environments with incomplete knowledge of state, and actions that are noisy and/or imprecise. This paper identifies a new problem sub-class that models settings in which information is revealed only intermittently through some exogenous process that provides state information periodically. Several practical domains fit this model, including the specific scenario that motivates our research: autonomous navigation of a planetary exploration rover augmented by remote imaging. With an eye to efficient specialized solution methods, we examine the structure of instances of this sub-class. They lead to Markov Decision Processes with exponentially large actionspaces but for which, as those actions comprise sequences of more atomic elements, one may establish performance bounds by comparing policies under different information assumptions. This provides a way in which to construct performance bounds systematically. Such bounds are useful because, in conjunction with the insights they confer, they can be employed in boundingbased methods to obtain high-quality solutions efficiently; the empirical results we present demonstrate their effectiveness for the considered problems. The foregoing has also alluded to the distinctive role that time plays for these problems -more specifically: time until information is revealed-and we uncover and discuss several interesting subtleties in this regard.

show abstract

Monte Carlo Tree Search: a review of recent modifications and applications

Cited by 103 publications

References 206 publications

Data-Driven Methods for Accelerating Polymer Design

Data-Driven Methods for Accelerating Polymer Design

Learning to Design Without Prior Data: Discovering Generalizable Design Strategies Using Deep Learning and Tree Search

Planning under periodic observations: bounds and bounding-based solutions

Contact Info

Product

Resources

About