Molecular contrastive learning of representations via graph neural networks

Wang, Yuyang; Wang, Jianren; Cao, Zhonglin; Farimani, Amir Barati

doi:10.1038/s42256-022-00447-x

“…D-MPNN [49] and AttentiveFP [50] are supervised GNNs methods. N-gram [51], PretrainGNN [22], GROVER [11], GraphMVP [26], MolCLR [12], and GEM [13] are pretraining methods. N-gram embeds the nodes in the graph and assembles them in short walks as the graph representation.…”

Section: Molecular Property Predictionmentioning

confidence: 99%

“…From the perspective of life science, the properties of molecules and the effects of drugs are mostly determined by their 3D structures [14, 15]. In most current MRL methods, one starts with representing molecules as 1D sequential strings, such as SMILES [16,17,18] and InChI [19,20,21], or 2D graphs [22,11,23,12,24]. This may limit their ability to incorporate 3D information for downstream tasks.…”

Section: Introductionmentioning

confidence: 99%

Uni-Mol: A Universal 3D Molecular Representation Learning Framework

Zhou

¹

,

Gao

²

,

Ding

³

et al. 2022

Preprint

View full text Add to dashboard Cite

Molecular representation learning (MRL) has gained tremendous attention due to its critical role in learning from limited supervised data for applications like drug design. In most MRL methods, molecules are treated as 1D sequential tokens or 2D topology graphs, limiting their ability to incorporate 3D information for downstream tasks and, in particular, making it almost impossible for 3D geometry prediction or generation. Herein, we propose Uni-Mol, a universal MRL framework that significantly enlarges the representation ability and application scope of MRL schemes. Uni-Mol is composed of two models with the same SE(3)-equivariant transformer architecture: a molecular pretraining model trained by 209M molecular conformations; a pocket pretraining model trained by 3M candidate protein pocket data. The two models are used independently for separate tasks, and are combined when used in protein-ligand binding tasks. By properly incorporating 3D information, Uni-Mol outperforms SOTA in 14/15 molecular property prediction tasks. Moreover, Uni-Mol achieves superior performance in 3D spatial tasks, including protein-ligand binding pose prediction, molecular conformation generation, etc. Finally, we show that Uni-Mol can be successfully applied to the tasks with few-shot data like pocket druggability prediction. The model and data will be made publicly available at \url{https://github.com/dptech-corp/Uni-Mol}

show abstract

“…As an alternative approach, contrastive learning has been applied to bias reduction [37] with the aim to learn a high-quality representation via a selfsupervised pretext task. This technique has been widely applied in various domains, including computer vision [38], graph data [39] and molecular data [40]. With an appropriate proposal distribution for negative sampling, this framework simultaneously improves the discriminative power of the model and reduces exposure bias.…”

Section: Related Workmentioning

confidence: 99%

3D pride without 2D prejudice: Bias-controlled multi-level generative models for structure-based ligand design

Chan¹,

Kumar²,

Verdonk³

et al. 2022

Preprint

View full text Add to dashboard Cite

Generative models for structure-based molecular design hold significant promise for drug discovery, with the potential to speed up the hit-to-lead development cycle, while improving the quality of drug candidates and reducing costs. Data sparsity and bias are, however, two main roadblocks to the development of 3D-aware models. Here we propose a first-in-kind training protocol based on multi-level contrastive learning for improved bias control and data efficiency. The framework leverages the large data resources available for 2D generative modelling with datasets of ligand-protein complexes. The result are hierarchical generative models that are topologically unbiased, explainable and customizable. We show how, by deconvolving the generative posterior into chemical, topological and structural context factors, we not only avoid common pitfalls in the design and evaluation of generative models, but furthermore gain detailed insight into the generative process itself. This improved transparency significantly aids method development, besides allowing fine-grained control over novelty vs familiarity.

show abstract

“…The chemical space that a drug candidate lies in is vast, while drug-related labeled data is limited. Not surprisingly, compared with traditional molecular fingerprint based models [9,10], recent molecular representation learning (MRL) models perform much better in most property prediction tasks [11,12,13]. However, to further improve the performance and extend the application scope of existing MRL models, one is faced with a critical issue.…”

Section: Introductionmentioning

confidence: 99%

“…From the perspective of life science, the properties of molecules and the effects of drugs are mostly determined by their 3D structures [14, 15]. In most current MRL methods, one starts with representing molecules as 1D sequential strings, such as SMILES [16,17,18] and InChI [19,20,21], or 2D graphs [22,11,23,12]. This may limit their ability to incorporate 3D information for downstream tasks.…”

Section: Introductionmentioning

confidence: 99%

Uni-Mol: A Universal 3D Molecular Representation Learning Framework

Zhou

¹

,

Gao

²

,

Ding

³

et al. 2022

Preprint

View full text Add to dashboard Cite

Molecular representation learning (MRL) has gained tremendous attention due to its critical role in learning from limited supervised data for applications like drug design. In most MRL methods, molecules are treated as 1D sequential tokens or 2D topology graphs, limiting their ability to incorporate 3D information for downstream tasks and, in particular, making it almost impossible for 3D geometry prediction or generation. Herein, we propose Uni-Mol, a universal MRL framework that significantly enlarges the representation ability and application scope of MRL schemes. Uni-Mol is composed of two models with the same SE(3)-equivariant transformer architecture: a molecular pretraining model trained by 209M molecular conformations; a pocket pretraining model trained by 3M candidate protein pocket data. The two models are used independently for separate tasks, and are combined when used in protein-ligand binding tasks. By properly incorporating 3D information, Uni-Mol outperforms SOTA in 14/15 molecular property prediction tasks. Moreover, Uni-Mol achieves superior performance in 3D spatial tasks, including protein-ligand binding pose prediction, molecular conformation generation, etc. Finally, we show that Uni-Mol can be successfully applied to the tasks with few-shot data like pocket druggability prediction. The model and data will be made publicly available at \url{https://github.com/dptech-corp/Uni-Mol}

show abstract

Molecular contrastive learning of representations via graph neural networks

Cited by 363 publications

References 40 publications

Uni-Mol: A Universal 3D Molecular Representation Learning Framework

Uni-Mol: A Universal 3D Molecular Representation Learning Framework

3D pride without 2D prejudice: Bias-controlled multi-level generative models for structure-based ligand design

Uni-Mol: A Universal 3D Molecular Representation Learning Framework

Contact Info

Product

Resources

About