Learning Attributed Graph Representation with Communicative Message Passing Transformer

Chen, Jianwen; Zheng, Shuangjia; Song, Ying; Rao, Jiahua; Yang, Yuedong

doi:10.24963/ijcai.2021/309

Cited by 36 publications

(44 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to fulfill the topology information of molecular graphs, some recent works [Kearnes et al ., 2016; Xiong et al ., 2020] attempt to apply GNNs to molecular representation learning. Besides, MPNN [Gilmer and others., 2017] and its variants DMPNN [Yang et al ., 2019], CMPNN [Song et al ., 2020], CoMPT [Chen et al ., 2021] utilize a message passing framework to better capture the inter-actions among atoms. However, they still require expensive annotations and barely generalize to unseen molecules, which pose hurdle to the practical applications.…”

Section: Related Workmentioning

confidence: 99%

Towards Effective and Generalizable Fine-tuning for Pre-trained Molecular Graph Models

Xia

Zheng

Tan

et al. 2022

Preprint

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) and Transformer have emerged as dominant tools for AI-driven drug discovery. Many state-of-the-art methods first pre-train GNNs or the hybrid of GNNs and Transformer on a large molecular database and then fine-tune on downstream tasks. However, different from other domains such as computer vision (CV) or natural language processing (NLP), getting labels for molecular data of downstream tasks often requires resource-intensive wet-lab experiments. Besides, the pre-trained models are often of extremely high complexity with huge parameters. These often cause the fine-tuned model to over-fit the training data of downstream tasks and significantly deteriorate the performance. To alleviate these critical yet underexplored issues, we propose two straightforward yet effective strategies to attain better generalization performance: 1. MolAug, which enriches the molecular datasets of downstream tasks with chemical homologies and enantiomers; 2. WordReg, which controls the complexity of the pre-trained models with a smoothness-inducing regularization built on dropout. Extensive experiments demonstrate that our proposed strategies achieve notable and consistent improvements over vanilla fine-tuning and yield multiple state-of-the-art results. Also, these strategies are model-agnostic and readily pluggable into fine-tuning of various pre-trained molecular graph models. We will release the code and the fine-tuned models.

show abstract

Section: Related Workmentioning

confidence: 99%

Towards Effective and Generalizable Fine-tuning for Pre-trained Molecular Graph Models

Xia

Zheng

Tan

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The global receptive fields of the self-attention and the local message passing of the graph neural network are inherently complementary and compatible. Attempts have been made in ways of incorporating graph information into the self-attention computation [35,39,40] and integrating the conventional graph neural networks [12] with Transformer architecture [11,4].…”

Section: Graph Transformermentioning

confidence: 99%

Retroformer: Pushing the Limits of Interpretable End-to-end Retrosynthesis Transformer

Wan¹,

Liao²,

Hsieh³

et al. 2022

Preprint

View full text Add to dashboard Cite

Retrosynthesis prediction is one of the fundamental challenges in organic synthesis. The task is to predict the reactants given a core product. With the advancement of machine learning, computer-aided synthesis planning has gained increasing interest. Numerous methods were proposed to solve this problem with different levels of dependency on additional chemical knowledge. In this paper, we propose Retroformer, a novel Transformer-based architecture for retrosynthesis prediction without relying on any cheminformatics tools for molecule editing. Via the proposed local attention head, the model can jointly encode the molecular sequence and graph, and efficiently exchange information between the local reactive region and the global reaction context. Retroformer reaches the new state-of-the-art accuracy for the end-to-end template-free retrosynthesis, and improves over many strong baselines on better molecule and reaction validity. In addition, its generative procedure is highly interpretable and controllable. Overall, Retroformer pushes the limits of the reaction reasoning ability of deep generative models.

show abstract

“…The prediction of molecular property has been widely considered as one of the most significant tasks in computational drug and material discovery (Goh, Hodas, and Vishnu 2017;Wu et al 2018;Chen et al 2018). Accurately predicting the property can help to evaluate and select the appropriate chemical molecules with desired characteristics for many downstream applications (Xiong et al 2019;Yang et al 2019;Song et al 2020;Chen et al 2021).…”

Section: Introductionmentioning

confidence: 99%

“…With the remarkable success of graph neural networks (GNNs) in various graph-related tasks in recent years (Wu et al 2020), a number of efforts have been made from different directions to design GNN models for molecular property prediction like (Yang et al 2019;Danel et al 2020;Maziarka et al 2020;Song et al 2020;Chen et al 2021). The fundamental idea is to regard the topology of atoms and bonds as a graph, and translate each molecule to a representation vector with powerful GNN encoders, followed by the prediction model for the specific property.…”

Section: Introductionmentioning

confidence: 99%

GeomGCL: Geometric Graph Contrastive Learning for Molecular Property Prediction

Zhou

et al. 2022

AAAI

View full text Add to dashboard Cite

Recently many efforts have been devoted to applying graph neural networks (GNNs) to molecular property prediction which is a fundamental task for computational drug and material discovery. One of major obstacles to hinder the successful prediction of molecular property by GNNs is the scarcity of labeled data. Though graph contrastive learning (GCL) methods have achieved extraordinary performance with insufficient labeled data, most focused on designing data augmentation schemes for general graphs. However, the fundamental property of a molecule could be altered with the augmentation method (like random perturbation) on molecular graphs. Whereas, the critical geometric information of molecules remains rarely explored under the current GNN and GCL architectures. To this end, we propose a novel graph contrastive learning method utilizing the geometry of the molecule across 2D and 3D views, which is named GeomGCL. Specifically, we first devise a dual-view geometric message passing network (GeomMPNN) to adaptively leverage the rich information of both 2D and 3D graphs of a molecule. The incorporation of geometric properties at different levels can greatly facilitate the molecular representation learning. Then a novel geometric graph contrastive scheme is designed to make both geometric views collaboratively supervise each other to improve the generalization ability of GeomMPNN. We evaluate GeomGCL on various downstream property prediction tasks via a finetune process. Experimental results on seven real-life molecular datasets demonstrate the effectiveness of our proposed GeomGCL against state-of-the-art baselines.

show abstract

Learning Attributed Graph Representation with Communicative Message Passing Transformer

Cited by 36 publications

References 1 publication

Towards Effective and Generalizable Fine-tuning for Pre-trained Molecular Graph Models

Towards Effective and Generalizable Fine-tuning for Pre-trained Molecular Graph Models

Retroformer: Pushing the Limits of Interpretable End-to-end Retrosynthesis Transformer

GeomGCL: Geometric Graph Contrastive Learning for Molecular Property Prediction

Contact Info

Product

Resources

About