Multilingual Molecular Representation Learning via Contrastive Pre-training

Zhang, Guo; Sharma, Pramod Kumar; Martinez, Andy; Li, Du; Abraham, Robin

doi:10.48550/arxiv.2109.08830

Cited by 4 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Early approaches to molecular representation learning predominantly focused on 1D SMILES (Wang et al, 2019; Chithrananda et al, 2020; Guo et al, 2021; Honda et al, 2019) and 2D graphs (Li et al, 2021; Lu et al, 2021; Fang et al, 2022b; Xia et al, 2022). Recently, there has been a growing interest in 3D molecular data, which could provide a more comprehensive reflection of physical properties, including information not captured by 1D and 2D data, such as conformation details.…”

Section: Related Workmentioning

confidence: 99%

“…Pre-training based molecular representation learning has shown remarkable performance across various molecular understanding tasks, such as drug discovery (Pinzi & Rastelli, 2019;Adelusi et al, 2022), molecular property prediction (Luo et al, 2022;Liu et al, 2022b;Zhou et al, 2023;Yu et al, 2023) and reaction prediction (Gastegger et al, 2021;Schwaller et al, 2021). Early approaches tend to model 1D SMILES (Wang et al, 2019;Guo et al, 2021;Honda et al, 2019) or 2D graphs (Li et al, 2021;Lu et al, 2021;Fang et al, 2022b;Xia et al, 2022). More recently, there has been a growing interest in 3D molecular data, with its inclusion of 3D structure information providing more comprehensive information of molecules.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Mol-AE: Auto-Encoder Based Molecular Representation Learning With 3D Cloze Test Objective

Yang,

Zheng,

Long

et al. 2024

Preprint

View full text Add to dashboard Cite

3D molecular representation learning has gained tremendous interest and achieved promising performance in various downstream tasks. A series of recent approaches follow a prevalent framework: an encoder-only model coupled with a coordinate denoising objective. Identifier, which should keep stable. The twisted optimization of these two roles is unstable. However, through a series of analytical experiments, we prove that the encoder-only model with coordinate denoising objective exhibits inconsistency between pre-training and downstream objectives, as well as issues with disrupted atomic identifiers. To address these two issues, we propose Mol-AE for molecular representation learning, an auto-encoder model using positional encoding as atomic identifiers. We also propose a new training objective named 3D Cloze Test to make the model learn better atom spatial relationships from real molecular substructures. Empirical results demonstrate that Mol-AE achieves a large margin performance gain compared to the current state-of-the-art 3D molecular modeling approach.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Mol-AE: Auto-Encoder Based Molecular Representation Learning With 3D Cloze Test Objective

Yang,

Zheng,

Long

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Transformers have been widely applied for molecular representation learning. However, many works either do not construct a readily available latent space for optimization, as in Uni-Mol [18], or construct a latent space, but do not implement a decoder for generating a molecule, as in KPGT [19], MM-Deacon [20], and GeoT [21]. We require a latent space, along with a decoder, to construct a decision space for optimization, along with the ability to generate a molecule from the vectorized latent representation.…”

Section: Related Workmentioning

confidence: 99%

Integrating Transformers and Many-Objective Optimization for Cancer Drug Design

Aksamit,

Hou,

et al. 2024

Preprint

View full text Add to dashboard Cite

Background: Drug design is a challenging and important task that requires the generation of novel and effective molecules that can bind to specific protein targets. Artificial intelligence (AI) algorithms have recently showed promising potential to expedite the drug design process. However, existing methods adopt multi-objective approaches which limits the number of objectives. Results: In this paper, we expand this thread of research from the many-objective perspective, by proposing a novel framework that integrates a latent Transformer-based model for molecular generation, with a drug design system that incorporates absorption, distribution, metabolism, excretion, and toxicity (ADMET) prediction, molecular docking, and many-objective metaheuristics. We compared the performance of two latent Transformer models (ReLSO and FragNet) on a molecular generation task and show that ReLSO outperforms FragNet in terms of reconstruction and latent space organization. We then explored six different many-objective metaheuristics based on evolutionary algorithms and particle swarm optimization on a drug design task involving potential drug candidates to human lysophosphatidic acid receptor 1 (LPA1), a cancer-related protein target. Conclusion: We show that multi-objective evolutionary algorithm based on dominance and decomposition (MOEA/DD) performs the best in terms of finding molecules that satisfy many objectives, such as high binding affinity and low toxicity, and high drug-likeness. Our framework demonstrates the potential of combining Transformers and many-objective computational intelligence for cancer drug design.

show abstract

Section: Related Workmentioning

confidence: 99%

Integrating transformers and many-objective optimization for drug design

Aksamit,

Hou,

et al. 2024

BMC Bioinformatics

View full text Add to dashboard Cite

Background Drug design is a challenging and important task that requires the generation of novel and effective molecules that can bind to specific protein targets. Artificial intelligence algorithms have recently showed promising potential to expedite the drug design process. However, existing methods adopt multi-objective approaches which limits the number of objectives. Results In this paper, we expand this thread of research from the many-objective perspective, by proposing a novel framework that integrates a latent Transformer-based model for molecular generation, with a drug design system that incorporates absorption, distribution, metabolism, excretion, and toxicity prediction, molecular docking, and many-objective metaheuristics. We compared the performance of two latent Transformer models (ReLSO and FragNet) on a molecular generation task and show that ReLSO outperforms FragNet in terms of reconstruction and latent space organization. We then explored six different many-objective metaheuristics based on evolutionary algorithms and particle swarm optimization on a drug design task involving potential drug candidates to human lysophosphatidic acid receptor 1, a cancer-related protein target. Conclusion We show that multi-objective evolutionary algorithm based on dominance and decomposition performs the best in terms of finding molecules that satisfy many objectives, such as high binding affinity and low toxicity, and high drug-likeness. Our framework demonstrates the potential of combining Transformers and many-objective computational intelligence for drug design.

show abstract

Multilingual Molecular Representation Learning via Contrastive Pre-training

Cited by 4 publications

References 0 publications

Mol-AE: Auto-Encoder Based Molecular Representation Learning With 3D Cloze Test Objective

Mol-AE: Auto-Encoder Based Molecular Representation Learning With 3D Cloze Test Objective

Integrating Transformers and Many-Objective Optimization for Cancer Drug Design

Integrating transformers and many-objective optimization for drug design

Contact Info

Product

Resources

About