Diagnostic Tools to Determine the Quality of “Transparent” Regression-Based QSARs:  The “Modelling Power” Plot

Abstract

A bivariate plot is presented for comparing two or more QSAR models. It is based on two new statistics associated with a regression model, the "descriptive power" (Dp), which is estimated through the relative uncertainty of model coefficients, and the "predictive power" (Pp), which is estimated through both the fitted and cross-validated explained variance of the response variable (i.e., biological activity). An algorithm was developed for performing equivalent multiple linear regression and partial-least-squares calculations. The results were validated by comparison with widely accepted commercial software. Dp and Pp statistics are defined to vary from 0 to 100%, so the modeler has a intuitive impression of the descriptive (i.e., global importance of the selected descriptors) and predictive (i.e., possibility of performing QSAR or just SAR estimations) power. These statistics represent a point in the Dp versus Pp "modelling power" plot, which facilitates visual multiple models comparison, but also could be used to substitute classical statistics and could even be combined to obtain a unique parameter to define (or compare) the model's quality.

Application of the modelling power approach to variable subset selection for GA-PLS QSAR models

“…However, even for slightly more sophisticated methods, such as leave-one-out (or leavemore-out) cross-validation, it has been demonstrated that these methods can be over-optimistic [4,5]. Recently, a more thorough and applicable method for evaluating the robustness of a QSAR was proposed-the Modelling Power Plot [6]. The Modelling Power Plot is a method to compare the quality of individual QSARs [6].…”

“…Recently, a more thorough and applicable method for evaluating the robustness of a QSAR was proposed-the Modelling Power Plot [6]. The Modelling Power Plot is a method to compare the quality of individual QSARs [6]. It is based on two new statistics associated with a regression model, the 'Descriptive power' (Dp), which is estimated through the relative uncertainty of model coefficients, and the 'Predictive power' (Pp), which is estimated through both the fitted and cross-validated explained variance of the response variable (i.e.…”

Chlorine Contribution to Quantitative Structure and Activity Relationship Models of Disinfection By-Products' Quantum Chemical Descriptors and Toxicities

“…Figure 5.5 Graphical comparison of models by the modeling power plot, based on the descriptive power (Dp) and the predictive power (Pp) Figure 5.5 shows the global modeling power plot for all the models. As can be observed from this figure, except model 5.6, there is a notable predictive ability for other QSAR models, if the criterion to make this conclusion is that Pp values must be approximately equal to, or larger than, 60% (Sagrado and Cronin, 2006). In contrast, all models have some descriptive ability except models 5.4 and 5.6 (Model 5.4 has the Dp= 58.25%, slightly smaller than 60%) using the same criterion based on 60% limit.…”

“…The equations and statistics for the QSAR models can be adjusted to the format suggested by Sagrado and Cronin (2006). Table 5.6 shows some conventional (r 2 , RMSE) and those used in this work (Dp, Pp, and Mp) statistics related to QSAR models in table 5.5.…”

“…Dp and Pp are diagnostic statistics (0-100% range) that reflect the descriptive and predictive power of the model, respectively. Mp (= f Dp D p + (1-f Dp ) P p ) represents the overall model quality; all models are selected independently on the f Dp value when it is equal to 0.5 (Sagrado and Cronin, 2006). Model 5.6 is of no interest in discrimination between wholly unacceptable models with negative Dp values.…”