Equivariant Graph Neural Networks for Toxicity Prediction

Abstract

Predictive modeling for toxicity is a crucial step in the drug discovery pipeline. It can help filter out molecules with a high probability of failing in the early stages of de novo drug design. Thus, several machine learning (ML) models have been developed to predict the toxicity of molecules by combining classical ML techniques or deep neural networks with well-known molecular representations such as fingerprints, InChi keys, SMILES strings, or 2D-graphs. Since molecules live in the physical 3D space, it is expected that their more natural and accurate representation would be defined in this geometric space as in ab initio methods for computing physicochemical properties. In such space, these molecules are also subject to geometric symmetries that only adequate 3D molecular representations can capture. Hence, the implications of using 3D structures in toxicity predictive modeling is an essential yet open question that needs further investigation. Recent studies showed great success in using equivariant graph neural networks (EGNNs) for representation learning based on 3D structures to predict potential energy surfaces and physicochemical properties of organic molecules and materials. Inspired by this, we investigate the performance of EG-NNs to construct reliable ML models for toxicity prediction. We use the equivariant transformer (ET) model in TorchMD-NET, an SE(3)-equivariant attention-weighted message-passing neural network. Eleven different toxicity datasets taken from Molecu-leNet, Therapeutic Data Commons (TDC), and ToxBenchmark have been considered to evaluate the capability of ET for toxicity prediction. Our results show that ET adequately learns 3D representations of molecules that can be successfully correlated with toxicity activity, achieving good accuracies on most datasets comparable to state-of-the-art 2D graph-based models and significantly better than a SMILES-based trans-former model. However, exhaustive (single- and multi-) conformational analysis has revealed a lack of correlation between the conformational samples and the predictive performance, which we find is only slightly affected by the conformer selection. Moreover, combining toxicity prediction in a multi-task setting with a neural network potential (energy prediction), or adding the total energy of the respective molecule as an additional node feature (besides atom types), does not benefit or even decrease the model's performance throughout all datasets. This suggests that a physicochemical property, such as the total energy, can not be related to toxicity activity with the selected ML method. We also provide an attention weight analysis for helping to explain the black-box toxicity prediction of our model in 3D space and, thus, increase reliability in ML models. In summary, our findings highlight the relevance of considering 3D geometry information in EGNNs and provide a straightforward way to integrate molecular conformers into ML-based pipelines for toxicity prediction.

Introduction to the Special Issue: AI Meets Toxicology

“…The methods described in this special issue cover a wide range of AI methods ranging from expert systems, ,, over similarity measures including read-across methods, − to classical machine learning such as random forests (RF), support vector machines (SVM), and artificial neural networks (ANN) − ,, to deep learning (DL) methods ,,− , including equivariant neural networks, deep generative models, and even large language models . In addition to models relying purely on the chemical structure, there is a notable trend of bringing in additional modalities to improve or inform predictive models. , In the following, we provide an overview of the AI approaches used in the publications contained in the SI.…”

Guided Docking as a Data Generation Approach Facilitates Structure-Based Machine Learning on Kinases

“…are the immediate neighbors of node i. Following related work, 4,53 the cutoff distance is set to d cut = 5 Å. For computational performance, we limit i ( ) to only include at × consist of categorical bond-types and features derived from interatomic distances.…”

Application of Transformers in Cheminformatics

“…With a similar SMILESbased model, transformer-CNN 110 predicts AMES mutagenicity and aqueous solubility. Cremer et al 111 applied TorchMD-NET, an equivariant graph transformer, on predicting drug toxicity.…”