OSR2Vec: an Ontology Semantic Representation Method Based Pre-training Natural Language Model

Liu, Yun; Sun, Huiting; Wang, Junjie; Hu, Jie; Feng, Wei; Cao, Xinyi

doi:10.21203/rs.3.rs-1379598/v1

Cited by 3 publications

(3 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We select a variety of GNN pre-training baselines, which include the latest state-of-the-art methods for molecular pre-training: EdgePred ( Hamilton et al 2017 ), InfoMax ( Veličković et al 2019 ), ContextPred ( Hu et al 2020 ), AttrMask ( Hu et al 2020 ), GraphCL ( You et al 2020 ), GraphMAE ( Hou et al 2022 ), redGraphMVP ( Liu et al 2022 ), JOAO ( You et al 2021 ), JOAOv2 ( You et al 2021 ), SimGRACE ( Xia et al 2022 ), and Mole-BERT ( Xia et al 2023a ). In addition, we also consider a new graph contrast learning module, TMCL, proposed in Mole-BERT.…”

Section: Methodsmentioning

confidence: 99%

MoleMCL: a multi-level contrastive learning framework for molecular pre-training

Zhang,

Xu,

Jiang

et al. 2024

Bioinformatics

View full text Add to dashboard Cite

Motivation Molecular representation learning plays an indispensable role in crucial tasks such as property prediction and drug design. Despite the notable achievements of Molecular Pre-training Models (MPMs), current methods often fail to capture both the structural and feature semantics of molecular graphs. Moreover, while graph contrastive learning has unveiled new prospects, existing augmentation techniques often struggle to retain their core semantics. To overcome these limitations, we propose a gradient-compensated encoder parameter perturbation approach, ensuring efficient and stable feature augmentation. By merging enhancement strategies grounded in attribute masking and parameter perturbation, we introduce MoleMCL, a new MOLEcular pre-training model based on Multi-level Contrastive Learning. Results Experimental results demonstrate that MoleMCL adeptly dissects the structure and feature semantics of molecular graphs, surpassing current state-of-the-art models in molecular prediction tasks, paving a novel avenue for molecular modeling. Availability and implementation The code and data underlying this work are available in GitHub at https://github.com/BioSequenceAnalysis/MoleMCL. Contact 23020221154148@stu.xmu.edu.cn Supplementary information Supplementary data are available at Bioinformatics online.

show abstract

Section: Methodsmentioning

confidence: 99%

MoleMCL: a multi-level contrastive learning framework for molecular pre-training

Zhang,

Xu,

Jiang

et al. 2024

Bioinformatics

View full text Add to dashboard Cite

show abstract

“…Nevertheless, the limited availability of labeled molecules and the vastness of the chemical space have constrained their prediction performance, particularly when handling out-of-distribution data samples 6,28,29 . Along with the remarkable achievements of selfsupervised learning methods in the fields of natural language processing 30,31 and computer vision 32,33 , these techniques have been employed to pre-train GNNs and improve the representation learning of molecules, leading to substantial improvements in the downstream molecular property prediction tasks 28,[34][35][36][37][38][39][40][41][42] .…”

mentioning

confidence: 99%

A knowledge-guided pre-training framework for improving molecular representation learning

Li,

Zhang,

Min

et al. 2023

Nat Commun

View full text Add to dashboard Cite

Learning effective molecular feature representation to facilitate molecular property prediction is of great significance for drug discovery. Recently, there has been a surge of interest in pre-training graph neural networks (GNNs) via self-supervised learning techniques to overcome the challenge of data scarcity in molecular property prediction. However, current self-supervised learning-based methods suffer from two main obstacles: the lack of a well-defined self-supervised learning strategy and the limited capacity of GNNs. Here, we propose Knowledge-guided Pre-training of Graph Transformer (KPGT), a self-supervised learning framework to alleviate the aforementioned issues and provide generalizable and robust molecular representations. The KPGT framework integrates a graph transformer specifically designed for molecular graphs and a knowledge-guided pre-training strategy, to fully capture both structural and semantic knowledge of molecules. Through extensive computational tests on 63 datasets, KPGT exhibits superior performance in predicting molecular properties across various domains. Moreover, the practical applicability of KPGT in drug discovery has been validated by identifying potential inhibitors of two antitumor targets: hematopoietic progenitor kinase 1 (HPK1) and fibroblast growth factor receptor 1 (FGFR1). Overall, KPGT can provide a powerful and useful tool for advancing the artificial intelligence (AI)-aided drug discovery process.

show abstract

“…1a, the 3D shape is a promising molecular similarity metrics that contains more pharmacological and physical information than the 2D structure and fingerprint similarity, therefore, 3D shape and pharmacological similarity is an optimal metric for contrastive learning. In GraphMVP 5 , GEM 6 , and Uni-Mol 7 , the authors primarily focused on 3D conformation as the pre-training task. In contrast, MolCLaSS 16 incorporates static 3D shape similarity into the framework of contrastive learning.…”

mentioning

confidence: 99%

Conformational Space Profile Enhances Generic Molecular Representation Learning

Wang,

Yang

et al. 2023

Preprint

View full text Add to dashboard Cite

The molecular representation model is a neural network that converts molecular representations (SMILES, Graph) into feature vectors, that carries the potential to be applied across a wide scope of drug discovery scenarios. However, current molecular representation models have been limited to 2D or static 3D structures, overlooking the dynamic nature of small molecules in solution and their ability to adopt flexible conformational changes crucial for drug-target interactions. To address this limitation, we propose a novel strategy that incorporates the conformational space profile into molecular representation learning. By capturing the intricate interplay between molecular structure and conformational space, our strategy enhances the representational capacity of our model named GeminiMol. Consequently, when pre-trained on a miniaturized molecular dataset, the GeminiMol model demonstrates a balanced and superior performance not only on traditional molecular property prediction tasks but also on zero-shot learning tasks, including virtual screening and target identification. By capturing the dynamic behavior of small molecules, our strategy paves the way for rapid exploration of chemical space, facilitating the transformation of drug design paradigms.

show abstract

OSR2Vec: an Ontology Semantic Representation Method Based Pre-training Natural Language Model

Cited by 3 publications

References 28 publications

MoleMCL: a multi-level contrastive learning framework for molecular pre-training

MoleMCL: a multi-level contrastive learning framework for molecular pre-training

A knowledge-guided pre-training framework for improving molecular representation learning

Conformational Space Profile Enhances Generic Molecular Representation Learning

Contact Info

Product

Resources

About