Data augmentation and transfer learning strategies for reaction prediction in low chemical data regimes

Zhang, Yun; Wang, Xinqiao; Zhang, Chengyun; Ge, Juan; Tang, Jing; Su, An; Duan, Hongliang

doi:10.1039/d0qo01636e

Cited by 49 publications

(41 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The model transfers beneficial information from the related task to aid its decision-making on the task with limited labels, resulting in improved performance. For example, transfer learning has enabled the molecular transformer to predict reaction outcomes with a small labelled dataset 22,23 . Transfer learning, however, still requires a large labelled dataset to train the related task, which often is not readily available.…”

Section: Introductionmentioning

confidence: 99%

“…For example, transfer learning has enabled the molecular transformer to predict reaction outcomes with a small labelled dataset. 22,23 Transfer learning, however, still requires a large labelled dataset to train the related task, which oen is not readily available. Actually, it is possible to initialize the feature detectors using reactions without any labels at all.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improving machine learning performance on small chemical reaction data with unsupervised contrastive pretraining

et al. 2022

View full text Add to dashboard Cite

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Improving machine learning performance on small chemical reaction data with unsupervised contrastive pretraining

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Molecular transformer, adapted from Vaswani's original transformer 8 , is the state-of-art SMILES-based seq2seq model 8,9 . Meanwhile, transfer learning has been equipped with the molecular transformer in the form of an additional fine-tuning step, and this combination is found to be beneficial for the forward prediction of complex reactions that involve regioselectivity and stereoselectivity such as carbohydrate reactions 10 , Heck reaction 11 , and Baeyer-Villiger reaction 12 . For example, the regio-and stereoselective carbohydrate reactions, a finetuning step with 20k carbohydrate reaction data gives a 30% increase in the prediction accuracy 10 .…”

Section: Introductionmentioning

confidence: 99%

Reproducing the Invention of a Named Reaction: Predicting Unseen Chemical Reactions via Zero-Shot Learning

Su¹,

Wang²,

Wang³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

<div> The study focuses on the proof-of-concept that the human invention of a named reaction can be reproduced by the zero-shot learning version of transformer.</div><div>While state-of-art reaction prediction machine learning models can predict chemical reactions through the transfer learning of thousands of training samples with the same reaction types as the ones to predict, how to prepare the models to predict truly "unseen" reactions remains a question. We aim to equip the transformer model with the ability to predict unseen reactions following the concept of "zero-shot learning". To find what kind of auxiliary information is needed, we reproduce the human invention of the Chan-Lam coupling reaction where the inventor was inspired by two existing reactions---Suzuki reaction and Barton's bismuth arylation reaction. After training with the samples from these two reactions as well as the USPTO dataset, the transformer model can pre-dict the Chan-Lam coupling reaction with 55.7% top-1 accuracy which is a huge im-provement comparing to 17.2% from the model trained with the USPTO dataset only. Our model also mimics the later stage of this history where the initial case of Chan-Lam coupling reaction was generalized to a wide range of reactants and reagents via the "one-shot learning" approach. The results of this study show that having existing reactions as auxiliary information can help the transformer predict unseen reactions and providing just one or few samples of the unseen reaction can boost the model's gener-alization ability.<br></div>

show abstract

“…[3][4][5] Scientists have explored chemical reactions from several fields, [6] including reaction prediction and retrosynthesis analysis. [7][8][9][10] Among others, reaction prediction, focusing on learning the rules of chemical reactions and then predicting the products with related properties from given reactants or reagents, has become an important topic in the past few years. The exists two mainstream models for reaction prediction are graph convolutional neural networks and simplified molecular-input lineentry system (SMILES)-based sequence-to-sequence (seq2seq) models.…”

Section: Introductionmentioning

confidence: 99%

From Theory to Experiment: Transformer-Based Generation Enables Rapid Discovery of Novel Reactions

Wang

Yao

Zhang

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Deep learning methods have been proven their potential roles in the chemical field, such as reaction prediction and retrosynthesis analysis. However, the de novo generation of unreported reactions using artificial intelligence technology remains not be completely explored. Inspired by molecular generation, we proposed the task of novel reaction generation. In this work, we applied the Heck reactions to train the transformer model, state-of-art natural language process model and obtained 4717 generated reactions after sampling and processing. We then confirmed that 2253 novel Heck reactions by organizing chemists to judge the generated reactions, and adopted organic synthesis experiment to verify the feasibility of unreported reactions. In this process, it only took 15 days from Heck reaction generation to experimental verification, proving that our model learns reaction rules in-depth and can make great contributions in the novel reaction discovery.

show abstract

Data augmentation and transfer learning strategies for reaction prediction in low chemical data regimes

Abstract: Effective and rapid deep learning method to predict chemical reactions contributes to the research and development of organic chemistry and drug discovery. Despite the outstanding capability of deep learning in...

Cited by 49 publications

References 34 publications

Improving machine learning performance on small chemical reaction data with unsupervised contrastive pretraining

Improving machine learning performance on small chemical reaction data with unsupervised contrastive pretraining

Reproducing the Invention of a Named Reaction: Predicting Unseen Chemical Reactions via Zero-Shot Learning

From Theory to Experiment: Transformer-Based Generation Enables Rapid Discovery of Novel Reactions

Contact Info

Product

Resources

About