Learning Retrosynthetic Planning through Simulated Experience

Abstract

The problem of retrosynthetic planning can be framed as a one-player game, in which the chemist (or a computer program) works backward from a molecular target to simpler starting materials through a series of choices regarding which reactions to perform. This game is challenging as the combinatorial space of possible choices is astronomical, and the value of each choice remains uncertain until the synthesis plan is completed and its cost evaluated. Here, we address this search problem using deep reinforcement learning to identify policies that make (near) optimal reaction choices during each step of retrosynthetic planning according to a user-defined cost metric. Using a simulated experience, we train a neural network to estimate the expected synthesis cost or value of any given molecule based on a representation of its molecular structure. We show that learned policies based on this value network can outperform a heuristic approach that favors symmetric disconnections when synthesizing unfamiliar molecules from available starting materials using the fewest number of reactions. We discuss how the learned policies described here can be incorporated into existing synthesis planning tools and how they can be adapted to changes in the synthesis cost objective or material availability.

Predicting retrosynthetic pathways using transformer-based models and a hyper-graph exploration strategy

“…9. Some of these are known compounds, for which the synthesis is reported in the literature (1,2,5,7,8), others (3,4,6,9) are unknown structures, which are similar to structures reported in the organic synthesis textbooks. The latter are challenging targets for students and useful to test the capability of the model to solve entirely new synthetic problems.…”

“…While human chemical knowledge will keep fueling the organic chemistry research in the years to come, a careful analysis of current trends 5,[7][8][9][10][11][12][13][14][15][16][17][18][19][20] and the application of basic extrapolation principles undeniably shows that there are growing expectations on the use of Articial Intelligence (AI) architectures to mimic human chemical intuition and to provide research assistant services to all bench chemists worldwide.…”

Reinforcement Learning for Bio-Retrosynthesis

“…Another major advance that could be included in RetroPath 3.0 would be to guide reaction selection steps through learned values instead of similarity. For example, this was implemented in (Segler, Preuss, and Waller 2018) or (Schreck, Coley, and Bishop 2019). However, the authors learned values from Reaxys (Elsevier Life Sciences n.d.) which contains 12.4 million single-step reactions (compared to around 80k in MetaNetX, including reactions without chemical structures (Moretti et al 2016)).…”

Predicting Retrosynthetic Pathways Using a Combined Linguistic Model and Hyper-Graph Exploration Strategy

“…An evaluation of the model was carried out through performing the retrosynthesis of the compounds reported in Figure 6. Some of these are known compounds, for which the synthesis is reported in literature (1,2,5,7,8), others are unknown structures (3,4,6,9). For the first group the evaluation of the model could be made by comparing the proposed retrosynthetic analysis with the known synthetic pathway.…”