Transfer learning for a foundational chemistry model

King-Smith, Emma

doi:10.1039/d3sc04928k

Cited by 3 publications

(4 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Kulik group are working towards the classification and discovery of hemilabile ligands in the CSD using the CSD Python API (Kevlishvili et al, 2023). Another recent endeavour has been the building of models via transfer learning for the prediction of common properties of interest (toxicity, odour and synthetic reaction yield) from 2D starting points, but incorporating inferred 3D information; this effort used organic molecules taken from the CSD as a useful training base for the model (King-Smith, 2024). It is intriguing that using 3D crystallographic information in this way can enhance the predictivity of models in seemingly far-removed areas of chemistry, perhaps demonstrating the importance of 3D structure in determining these properties and the need for such information to be represented in quantitative structure-property relationship (QSPR) models more generally.…”

Section: Focused Predictive Model Buildingmentioning

confidence: 99%

What has scripting ever done for us? The CSD Python application programming interface (API)

Sykes,

Johnson,

Kingsbury

et al. 2024

J Appl Crystallogr

View full text Add to dashboard Cite

Since its first release in 2016, the Cambridge Structural Database Python application programming interface (CSD Python API) has seen steady uptake within the community that the Cambridge Crystallographic Data Centre serves. This article reviews the history of scripting interfaces, demonstrating the need, and then briefly outlines the technical structure of the API. It describes the reach of the CSD Python API, provides a selected review of its impact and gives some illustrative examples of what scientists can do with it. The article concludes with speculation as to how such endeavours will evolve over the next decade.

show abstract

Section: Focused Predictive Model Buildingmentioning

confidence: 99%

What has scripting ever done for us? The CSD Python application programming interface (API)

Sykes,

Johnson,

Kingsbury

et al. 2024

J Appl Crystallogr

View full text Add to dashboard Cite

show abstract

“…The molecular representation is the same as the one used for pretraining, the encoder can be either finetuned or kept frozen during training, and a dedicated decoder is trained for each downstream task. (C) The transfer learning architecture used in King-Smith's work 37.…”

mentioning

confidence: 99%

“…The recent work by King-Smith proposes a machine-learning framework that leverages graph-based MRL to make accurate predictions in chemistry-related tasks with limited data. 37 The approach involves pretraining a graph neural network model on a dataset of organic crystal structures from the Cambridge Crystallographic Data Centre (CCDC). 28 In the pretraining phase, a supervised-learning approach was employed.…”

mentioning

confidence: 99%

See 1 more Smart Citation

A focus on molecular representation learning for the prediction of chemical properties

Harnik,

Milo

2024

Chem. Sci.

View full text Add to dashboard Cite

show abstract

Transfer learning across different chemical domains: virtual screening of organic materials with deep learning models pretrained on small molecule and chemical reaction data

Zhang,

Zhai,

Gong

et al. 2024

J Cheminform

View full text Add to dashboard Cite

Machine learning is becoming a preferred method for the virtual screening of organic materials due to its cost-effectiveness over traditional computationally demanding techniques. However, the scarcity of labeled data for organic materials poses a significant challenge for training advanced machine learning models. This study showcases the potential of utilizing databases of drug-like small molecules and chemical reactions to pretrain the BERT model, enhancing its performance in the virtual screening of organic materials. By fine-tuning the BERT models with data from five virtual screening tasks, the version pretrained with the USPTO–SMILES dataset achieved R 2 scores exceeding 0.94 for three tasks and over 0.81 for two others. This performance surpasses that of models pretrained on the small molecule or organic materials databases and outperforms three traditional machine learning models trained directly on virtual screening data. The success of the USPTO–SMILES pretrained BERT model can be attributed to the diverse array of organic building blocks in the USPTO database, offering a broader exploration of the chemical space. The study further suggests that accessing a reaction database with a wider range of reactions than the USPTO could further enhance model performance. Overall, this research validates the feasibility of applying transfer learning across different chemical domains for the efficient virtual screening of organic materials. Scientific contribution This study verifies the feasibility of applying transfer learning to large language models in different chemical fields to help organic materials perform virtual screening. Through the comparison of transfer learning from different chemical fields to a variety of organic material molecules, the high precision virtual screening of organic materials is realized. Supplementary Information The online version contains supplementary material available at 10.1186/s13321-024-00886-1.

show abstract

Transfer learning for a foundational chemistry model

Abstract: Harnessing knowledge from crystal structures yields a model that can predict a variety of chemistry-relevant outcomes.

Cited by 3 publications

References 44 publications

What has scripting ever done for us? The CSD Python application programming interface (API)

What has scripting ever done for us? The CSD Python application programming interface (API)

A focus on molecular representation learning for the prediction of chemical properties

Transfer learning across different chemical domains: virtual screening of organic materials with deep learning models pretrained on small molecule and chemical reaction data

Contact Info

Product

Resources

About