A knowledge-guided pre-training framework for improving molecular representation learning

Abstract

Learning effective molecular feature representation to facilitate molecular property prediction is of great significance for drug discovery. Recently, there has been a surge of interest in pre-training graph neural networks (GNNs) via self-supervised learning techniques to overcome the challenge of data scarcity in molecular property prediction. However, current self-supervised learning-based methods suffer from two main obstacles: the lack of a well-defined self-supervised learning strategy and the limited capacity of GNNs. Here, we propose Knowledge-guided Pre-training of Graph Transformer (KPGT), a self-supervised learning framework to alleviate the aforementioned issues and provide generalizable and robust molecular representations. The KPGT framework integrates a graph transformer specifically designed for molecular graphs and a knowledge-guided pre-training strategy, to fully capture both structural and semantic knowledge of molecules. Through extensive computational tests on 63 datasets, KPGT exhibits superior performance in predicting molecular properties across various domains. Moreover, the practical applicability of KPGT in drug discovery has been validated by identifying potential inhibitors of two antitumor targets: hematopoietic progenitor kinase 1 (HPK1) and fibroblast growth factor receptor 1 (FGFR1). Overall, KPGT can provide a powerful and useful tool for advancing the artificial intelligence (AI)-aided drug discovery process.

Deep representation learning of chemical-induced transcriptional profile for phenotype-based drug discovery

“…Initializing TranSiGen with perturbational profiles generated by gene knockdown yields superior performance compared to random initialization. Additionally, using pre-training representation, Knowledge-guided Pre-training of Graph Transformer (KPGT) 21 , further enhances the performance of inferring DEGs, surpassing the molecular fingerprint ECFP4 (as detailed in the Molecular representations in Method and corroborated by the metric scores in Supplementary Tables 2 and 3 ).…”

“…Considering that the current number of compounds with experimentally measured gene expression profiles is still limited compared to the vast chemical space, TranSiGen utilized the pre-trained molecular representation KPGT 21 for compounds. KPGT is a novel self-supervised learning framework for molecular graph representation.…”

Quantum-Informed Molecular Representation Learning Enhancing ADMET Property Prediction

“…Injecting such domain knowledge into pretraining has not received much attention, although it is a straightforward strategy and domain-related property predictions are popularly taken as downstream tasks. KPGT attempted to utilize node-wise chemical properties in pretraining methods, resulting in significantly enhanced performance. Our pretraining technique follows in the footsteps of these efforts, demonstrating the potential of domain-knowledge-guided pretraining.…”

Triple Generative Self-Supervised Learning Method for Molecular Property Prediction

“…Graph contrastive coding (GCC) [28] designs a self-supervised graph neural network pre-training framework to capture common network topological properties across multiple networks. The KPGT [29] self-supervised framework introduces the line graph transformer (LiGhT), which is mainly used to accurately simulate the structural information of molecular graphs. However, it ignores the unique structural properties of chemical molecules, such as rings and functional groups.…”