The Importance of Being Earnest: Validation is the Absolute Essential for Successful Application and Interpretation of QSPR Models

Abstract

This paper emphasizes the importance of rigorous validation as a crucial, integral component of Quantitative Structure Property Relationship (QSPR) model development. We consider some examples of published QSPR models, which in spite of their high fitted accuracy for the training sets and apparent mechanistic appeal, fail rigorous validation tests, and, thus, may lack practical utility as reliable screening tools. We present a set of simple guidelines for developing validated and predictive QSPR models. To this end, we discuss several validation strategies including (1) randomization of the modelled property, also called Y-scrambling, (2) multiple leave-many-out crossvalidations, and (3) external validation using rational division of a dataset into training and test sets. We also highlight the need to establish the domain of model applicability in the chemical space to flag molecules for which predictions may be unreliable, and discuss some algorithms that can be used for this purpose. We advocate the broad use of these guidelines in the development of predictive QSPR models.

Investigation of substituent effect of 1-(3,3-diphenylpropyl)-piperidinyl phenylacetamides on CCR5 binding affinity using QSAR and virtual screening techniques

“…The proposed method, due to the high predictive ability [17,27], can provide a useful aid to the costly and time consuming experiments for determining the CCR5 binding affinity. The method can also be used to screen existing databases or virtual libraries in order to identify new potentially active compounds.…”

“…This technique ensures the robustness of a QSAR model [16,17]. The dependent variable vector (biological action) is randomly shuffled and a new QSAR model is developed using the original independent variable matrix.…”

“…Furthermore Tropsha's research group [17,18] considered a QSAR model predictive, if the following conditions are satisfied: …”

“…The domain of application [17,19] of a QSAR model must be defined if the model is to be used for screening new compounds. Predictions for only those compounds that fall into this domain may be considered reliable.…”

“…Predictions for only those compounds that fall into this domain may be considered reliable. Extent of Extrapolation [17] is one simple approach to define the applicability of the domain. It is based on the calculation of the leverage h i [20] for each chemical, where the QSAR model is used to predict its activity:…”