State-of-the-art augmented NLP transformer models for direct and single-step retrosynthesis

Abstract

We investigated the effect of different training scenarios on predicting the (retro)synthesis of chemical compounds using text-like representation of chemical reactions (SMILES) and Natural Language Processing (NLP) neural network Transformer architecture. We showed that data augmentation, which is a powerful method used in image processing, eliminated the effect of data memorization by neural networks and improved their performance for prediction of new sequences. This effect was observed when augmentation was used simultaneously for input and the target data simultaneously. The top-5 accuracy was 84.8% for the prediction of the largest fragment (thus identifying principal transformation for classical retro-synthesis) for the USPTO-50k test dataset, and was achieved by a combination of SMILES augmentation and a beam search algorithm. The same approach provided significantly better results for the prediction of direct reactions from the single-step USPTO-MIT test set. Our model achieved 90.6% top-1 and 96.1% top-5 accuracy for its challenging mixed set and 97% top-5 accuracy for the USPTO-MIT separated set. It also significantly improved results for USPTO-full set single-step retrosynthesis for both top-1 and top-10 accuracies. The appearance frequency of the most abundantly generated SMILES was well correlated with the prediction outcome and can be used as a measure of the quality of reaction prediction.

Substructure-based neural machine translation for retrosynthetic prediction

“…According to reproduced results presented by Lin et al [37], Top-1 accuracy ranges from 28.3% (Liu et al [34] LSTM model over the USPTO 50 K dataset) to 54.1% (Transformer model over the USPTO MIT dataset by Lin et al [37]). In the most recent report by Tetko et al [46], an augmented Transformer model has reached Top-1 accuracy of 53.5% trained with 100 times augmented USPTO-50 K dataset with beam size…”

“…The diversity of reactant candidates is one of the important aspects of a retrosynthesis prediction. In the recently published paper [46], the diversity of the reactant candidates is discussed within the context of top-5 performance analysis. One of the goals of a retrosynthetic model is to obtain multiple precursor suggestions, and the top-N approach may suggest other probable reactant candidates.…”

“…Recently, the best performing NMT models include an attention mechanism [40,43] as a part of their neural architectures to enhance their performances on longer sentences [27,[32][33][34]. There are also retrosynthetic predictors built on the Transformer architecture [31,37,[44][45][46], based solely on the attention mechanism. Encoder-decoder models, especially once an attention mechanism is introduced, all employ similar strategies to handle a translation task.…”

Predicting Enzymatic Reactions with a Molecular Transformer

“…Additional examples reflect inaccurate database entries. In the hydrolysis of a βhydroxysulfone by porcine liver esterase, 45 the Enzymatic Transformer correctly predicts the alcohol hydrolysis product, however this product is unstable and spontaneously eliminates to form a styrene, which is the product isolated and recorded in the database (reaction (11)). The Enzymatic Transformer also correctly predicts the formation of thymine from the hydrolysis of a thymidine nucleoside analog by uridine phosphorylase, 46 however the database entry wrongly recorded the isomeric 6-methyl-uracil as the product (reaction (12)…”

From Big Data to Artificial Intelligence: chemoinformatics meets new challenges

“…The benchmarking studies [ 20 , 22 ] show that such methods can achieve similar or better performances compared to traditional methods for classical tasks such as QSAR[ 24 ] but at the same time allow for intuitive interpretation of models [ 21 ]. Moreover, they can be used to address very different tasks, such as the aforementioned generation of molecules with desired properties or/and the prediction of single step (retro) synthesis [ 25 , 26 ], or even complete retro-synthesis [ 27 , 28 ] that could not be achieved with traditional methods. All these approaches are part of the emerging area of AI, which is going to drive the future of chemoinformatics.…”