Comparative Studies on Some Metrics for External Validation of QSPR Models

Abstract

Quantitative structure-property relationship (QSPR) models used for prediction of property of untested chemicals can be utilized for prioritization plan of synthesis and experimental testing of new compounds. Validation of QSPR models plays a crucial role for judgment of the reliability of predictions of such models. In the QSPR literature, serious attention is now given to external validation for checking reliability of QSPR models, and predictive quality is in the most cases judged based on the quality of predictions of property of a single test set as reflected in one or more external validation metrics. Here, we have shown that a single QSPR model may show a variable degree of prediction quality as reflected in some variants of external validation metrics like Q²(F1), Q²(F2), Q²(F3), CCC, and r²(m) (all of which are differently modified forms of predicted variance, which theoretically may attain a maximum value of 1), depending on the test set composition and test set size. Thus, this report questions the appropriateness of the common practice of the "classic" approach of external validation based on a single test set and thereby derives a conclusion about predictive quality of a model on the basis of a particular validation metric. The present work further demonstrates that among the considered external validation metrics, r²(m) shows statistically significantly different numerical values from others among which CCC is the most optimistic or less stringent. Furthermore, at a given level of threshold value of acceptance for external validation metrics, r²(m) provides the most stringent criterion (especially with Δr²(m) at highest tolerated value of 0.2) of external validation, which may be adopted in the case of regulatory decision support processes.

In silico designing of power conversion efficient organic lead dyes for solar cells using todays innovative approaches to assure renewable energy for future

“…[42][43][44] The robustness of each QSPR model was checked by the Y-randomization technique to classify whether the model was obtained coincidentally or not. The model randomization was computed 100 times through rearranging the response while retaining the original independent variables or descriptor matrix.…”

Coral: QSPR modeling of rate constants of reactions between organic aromatic pollutants and hydroxyl radical

“…Thus, the aforementioned principles (a-g) are in agreement with OECD principles. [20] It should be noted that suggested models for log(OH) are satisfactory from point of view of new criteria suggested by Roy et al: [21] for all four models (Table 2) r 2 m > 0:5 and Dr 2 m < 0:2. The results of this study are better than in our previous study.…”

Quimioinformática: Uma Introdução

“…104 Apesar de necessária, essa métrica é insuficiente para se avaliar a preditividade externa de um modelo, 101 por isso é necessário se avaliar o coeficiente de correlação de validação cruzada externa (Q 2 ), que é considerado válido quando Q 2 ≥ 0.6. 104 Os modelos categóricos são validados usando-se a 110,111 Por fim, vale mencionar a técnica de aleatorização da variável Y (Y-randomization), que é recomendada para se garantir que os resultados dos modelos de QSAR não sejam provenientes do acaso. Nesse procedimento, a variável Y é aleatorizada e novos modelos são gerados.…”