Transparency in Modeling through Careful Application of OECD’s QSAR/QSPR Principles via a Curated Water Solubility Data Set

Abstract

The need for careful assembly, training, and validation of quantitative structure−activity/property models (QSAR/QSPR) is more significant than ever as data sets become larger and sophisticated machine learning tools become increasingly ubiquitous and accessible to the scientific community. Regulatory agencies such as the United States Environmental Protection Agency must carefully scrutinize each aspect of a resulting QSAR/QSPR model to determine its potential use in environmental exposure and hazard assessment. Herein, we revisit the goals of the Organisation for Economic Cooperation and Development (OECD) in our application and discuss the validation principles for structure−activity models. We apply these principles to a model for predicting water solubility of organic compounds derived using random forest regression, a common machine learning approach in the QSA/PR literature. Using public sources, we carefully assembled and curated a data set consisting of 10,200 unique chemical structures with associated water solubility measurements. This data set was then used as a focal narrative to methodically consider the OECD's QSA/PR principles and how they can be applied to random forests. Despite some expert, mechanistically informed supervision of descriptor selection to enhance model interpretability, we achieved a model of water solubility with comparable performance to previously published models (5-fold cross validated performance 0.81 R 2 and 0.98 RMSE). We hope this work will catalyze a necessary conversation around the importance of cautiously modernizing and explicitly leveraging OECD principles while pursuing state-of-the-art machine learning approaches to derive QSA/PR models suitable for regulatory consideration.

Will we ever be able to accurately predict solubility?

“…Our study corroborates the findings of Lowe et al . 29 , emphasizing the complexity and challenges in solubility prediction across diverse chemical spaces. We found that RF models provide a balanced and interpretable framework.…”

“… 28 , and Lowe et al . 29 . In comparison, AqSolDB which was published in 2020 has already been used in 2021 by Francoeur et al .…”

“… 33 , and in 2023 by Lowe et al . 29 . AqSolDB is one of the largest publicly accessible set with 9,982 entries.…”

“… 33 2D-Graph & Molecular FP 12,849 AqSolDB, ONSC, AAT & BNNLap LightGBM 0.96 0.8 Lowe et al . 29 PaDEL 8,037 ADDoPT, AqSolDB, Bradley, eChemPortalAPI, LookChem, OChem, OPERA, PubChem, QSARDB RF 0.97 0.82 …”

Introduction to the Special Issue: AI Meets Toxicology

“…The tasks and problems spanned by the proposed methods are also highly diverse and range from classical quantitative structure–activity/property relationship (QSAR/QSPR) problems, such as the prediction of drug-induced liver injury (DILI), , Ames mutagenicity, , or acetylcholinesterase (AChE) inhibition, to categorization of chemicals, generating transcriptomic profiles and classifying parts of regulatory documents . In their paper, “Transparency in Modeling through Careful Application of OECD’s QSAR/QSPR Principles via a Curated Water Solubility Data Set”, the authors revisited the five Organisation for Economic Co-operation and Development (OECD) principles for QSAR models. The discussion of these principles, which is ongoing also within OECD, is addressed here with particular emphasis on the case of machine learning (ML) models.…”