Retrosynthetic planning with experience-guided Monte Carlo tree search

Abstract

In retrosynthetic planning, the huge number of possible routes to synthesize a complex molecule using simple building blocks leads to a combinatorial explosion of possibilities. Even experienced chemists often have difficulty to select the most promising transformations. The current approaches rely on human-defined or machine-trained score functions which have limited chemical knowledge or use expensive estimation methods for guiding. Here we propose an experience-guided Monte Carlo tree search (EG-MCTS) to deal with this problem. Instead of rollout, we build an experience guidance network to learn knowledge from synthetic experiences during the search. Experiments on benchmark USPTO datasets show that, EG-MCTS gains significant improvement over state-of-the-art approaches both in efficiency and effectiveness. In a comparative experiment with the literature, our computer-generated routes mostly matched the reported routes. Routes designed for real drug compounds exhibit the effectiveness of EG-MCTS on assisting chemists performing retrosynthetic analysis.

AiZynthFinder 4.0: developments based on learnings from 3 years of industrial application

“…In order to improve the lead time for synthesis, one could do an analysis of the most frequently used externally available building blocks and make sure that they are available at AstraZeneca’s internal storages. For the ChEMBL and GDB target sets, we used a combination of stocks that we created for the first release of AiZynthFinder from the ZINC database, and the E-Molecules building blocks, a popular choice in multi-step retrosynthesis publications [ 40 , 50 ]. We see in Fig.…”

“…Before we offer insight into the newly implemented features and structure, we provide a concise overview of the previously implemented algorithm [ 23 ]: The retrosynthesis process is carried out by taking an input target molecule to decompose into purchasable precursors. The default search algorithm used is the Monte Carlo tree search (MCTS) [ 39 ] that together with a neural network-based policy is used to predict routes [ 40 ]. This is accomplished by iteratively expanding promising nodes in the tree search by applying reaction templates.…”

“…Apart from offering speed-ups, moving to use ONNX as the ML back-end for the template-based models allows us to be model-agnostic, applying template-based models trained by other groups in AiZynthFinder. For instance, we converted the PyTorch-based model trained by Chen et al [ 50 ] that has been used in several publications [ 40 ] and compared it to our UPSTO-based model [ 8 ] (see Supporting Information for further information).…”

Retrosynthesis Zero: Self-Improving Global Synthesis Planning Using Reinforcement Learning

“…Many retrosynthetic planning algorithms exploit reinforcement learning for more efficient route searching. For instance, EG-MCTS, 33 a similar method of learning from MCTS, uses synthetic experiences generated by a pretrained policy neural network and MCTS to learn a value function. The value function assists the MCTS process in avoiding unfavorable actions that easily lead to hard-to-solve molecules, and prevents excessive focus on actions with lower predicted probabilities.…”

Exploring Chemical Reaction Space with Machine Learning Models: Representation and Feature Perspective

“…By contrast, molecular graphs can preserve the chemically stable groups undisturbed. Therefore, molecular graphs find extensive use in tasks involving multistep structural changes, especially retrosynthetic planning. , Besides, graph edits are also applied for the single-step retrosynthesis task, where target molecules are transformed into reactants by applying an iterative refining procedure. For example, Igashov et al generate the reactants for the target molecule by applying diffusion models on the graph level.…”