Assessment and Reproducibility of Quantitative Structure–Activity Relationship Models by the Nonexpert

Abstract

Model reliability is generally assessed and reported as an intrinsic component of quantitative structure-activity relationship (QSAR) publications; it can be evaluated using defined quality criteria such as the Organisation for Economic Cooperation and Development (OECD) principles for the validation of QSARs. However, less emphasis is afforded to the assessment of model reproducibility, particularly by users who may wish to use model outcomes for decision making, but who are not QSAR experts. In this study we identified a range of QSARs in the area of absorption, distribution, metabolism, and elimination (ADME) prediction and assessed their adherence to the OECD principles, as well as investigating their reproducibility by scientists without expertise in QSAR. Here, 85 papers were reviewed, reporting over 80 models for 31 ADME-related endpoints. Of these, 12 models were identified that fulfilled at least 4 of the 5 OECD principles and 3 of these 12 could be readily reproduced. Published QSAR models should aim to meet a standard level of quality and be clearly communicated, ensuring their reproducibility, to progress the uptake of the models in both research and regulatory landscapes. A pragmatic workflow for implementing published QSAR models and recommendations to modellers, for publishing models with greater usability, are presented herein.

Towards reproducible computational drug discovery

“…Although, with the availability of ample literature on the best practice in QSAR modeling [155], it is mostly aimed at those having cheminformatics/mathematical understanding of the subject. In a recently published article, Patel et al [156] assessed the reproducibility of QSAR models pertaining to ADME predictions by scientists without expertise in QSAR. The authors reviewed 85 papers spanning 80 models with ADME related endpoints and presented a pragmatic workflow for the implementation of QSAR models with greater usability.…”

QSPR/QSAR: State-of-Art, Weirdness, the Future

“…However, some "unpleasant peculiarities" remain. The list of "main unpleasant peculiarities" of QSPR/QSAR analysis is as follows: (i) possibility of "chance correlations" [43][44][45][46]; (ii) possibility of overtraining [47]; (iii) possibility of weak reproducibility of statistical quality of an approach suggested [48,49].…”

Advances in the prediction of gastrointestinal absorption: Quantitative Structure-Activity Relationship (QSAR) modelling of PAMPA permeability

“…In addition, the models developed are useful for screening purposes and also provide an insight into the structural characteristics important for better absorption thus guiding the design of new bioactive compounds with desirable ADME properties. Whilst many models exist, there is no overall consensus over the optimal modelling approach and which conclusions may be drawn [13].…”