Trade-off Predictivity and Explainability for Machine-Learning Powered Predictive Toxicology: An in-Depth Investigation with Tox21 Data Sets

Abstract

Selecting a model in predictive toxicology often involves a trade-off between prediction performance and explainability: should we sacrifice the model performance to gain explainability or vice versa. Here we present a comprehensive study to assess algorithm and feature influences on model performance in chemical toxicity research. We conducted over 5000 models for a Tox21 bioassay data set of 65 assays and ∼7600 compounds. Seven molecular representations as features and 12 modeling approaches varying in complexity and explainability were employed to systematically investigate the impact of various factors on model performance and explainability. We demonstrated that end points dictated a model's performance, regardless of the chosen modeling approach including deep learning and chemical features. Overall, more complex models such as (LS-)SVM and Random Forest performed marginally better than simpler models such as linear regression and KNN in the presented Tox21 data analysis. Since a simpler model with acceptable performance often also is easy to interpret for the Tox21 data set, it clearly was the preferred choice due to its better explainability. Given that each data set had its own error structure both for dependent and independent variables, we strongly recommend that it is important to conduct a systematic study with a broad range of model complexity and feature explainability to identify model balancing its predictivity and explainability.

A review of explainable and interpretable AI with applications in COVID‐19 imaging

“…In general, a tradeoff exists between the complexity/depth of an AI system and its interpretability, with classical, shallow algorithms, such as decision trees, providing more explainable output with a potentially reduced performance. 34,37,38 Figure 2 depicts this phenomenon for several commonly used algorithms. It is important to note that finding the optimal operating point between system performance,which will improve patient management, and system interpretability, which will lead to more frequent implementation and trust in radiological practice, is critical.…”

“…In general, a tradeoff exists between the complexity/depth of an AI system and its interpretability, with classical, shallow algorithms, such as decision trees, providing more explainable output with a potentially reduced performance 34,37,38 . Figure 2 depicts this phenomenon for several commonly used algorithms.…”

Class imbalance learning with Bayesian optimization applied in drug discovery

“…Machine intelligence (MI), including machine learning and deep learning, has been successfully applied to drug discovery and is regarded as a promising method for such candidate selection 6 , 7 . However, there is a dilemma between performance and interpretability within MI 8 , which has limited its application. Previous studies have shown that deep learning models perform better than machine learning models on classifications 9 , 10 but are harder to interpret 11 .…”

TargetTox: A Feature Selection Pipeline for Identifying Predictive Targets Associated with Drug Toxicity

“…Many factors can affect the performance of QSAR models, including the chemical features used, the supervised learning algorithm employed, the composition of training set, and the endpoint of interest. 8 QSAR models proved effective in predicting wellestablished in vitro toxicity endpoints 9 but often fell short of accuracy when used to predict complex in vivo endpoints such as drug adverse events. 10 This can be attributed to the relatively low structural similarity among drugs causing the same adverse events, 11 as well as the largely uncharacterized pharmacokinetics processes drugs undergo in vivo.…”