Could Graph Neural Networks Learn Better Molecular Representation for Drug Discovery? A Comparison Study of Descriptor-based and Graph-based Models

Abstract

Graph neural networks (GNN) has been considered as an attractive modelling method for molecular property prediction, and numerous studies have shown that GNN could yield more promising results than traditional descriptor-based methods. In this study, based on 11 public datasets covering various property endpoints, the predictive capacity and computational efficiency of the prediction models developed by eight machine learning (ML) algorithms, including four descriptor-based models (SVM, XGBoost, RF and DNN) and four graph-based models (GCN, GAT, MPNN and Attentive FP), were extensively tested and compared. The results demonstrate that on average the descriptor-based models outperform the graph-based models in terms of prediction accuracy and computational efficiency. SVM generally achieves the best predictions for the regression tasks. Both RF and XGBoost can achieve reliable predictions for the classification tasks, and some of the graph-based models, such as Attentive FP and GCN, can yield outstanding performance for a fraction of larger or multi-task datasets. In terms of computational cost, XGBoost and RF are the two most efficient algorithms and only need a few seconds to train a model even for a large dataset. The model interpretations by the SHAP method can effectively explore the established domain knowledge for the descriptor-based models. Finally, we explored use of these models for virtual screening (VS) towards HIV and demonstrated that different ML algorithms offer diverse VS profiles. All in all, we believe that the off-the-shelf descriptor-based models still can be directly employed to accurately predict various chemical endpoints with excellent computability and interpretability.

Siamese Recurrent Neural Network with a Self-Attention Mechanism for Bioactivity Prediction

“…To benchmark the deep model, two popular machine learning methods, 8 , 33 RF and SVM, were implemented with scikit-learn. The two methods were trained on the same training sets for the binary classification but taking a single compound as input using either the ECFP6 fingerprints or the tokenized SMILES strings.…”

“…Both RF and SVM prove to consistently perform well on a variety of tasks. 8 , 33 For the RF, the number of trees was set to 100, and no maximum depth for the tree was specified. The Gini Index for information gain was used.…”

Natural product scores and fingerprints extracted from artificial neural networks

“…This falls in line with earlier research of ours [12] , and is not uncommon to occur. For example, Jiang and colleagues [34] showed that the Attention FP [35] , a graph neural network providing state of the art results on many benchmark sets, performs only on par with descriptor-based models. The results of the GCN can be found in the Supplementary Information .…”

State of the Art Iterative Docking with Logistic Regression and Morgan Fingerprints

“…These results come at a time when graph-based neural networks are increasingly popular for computational chemistry, although their benefits have also come into question (29). The performance of the de facto standard 2,048 bit Morgan fingerprint can be improved simply by using pharmacophoric atom invariants and/or larger fingerprint sizes.…”