Study of the Applicability Domain of the QSAR Classification Models by Means of the Rivality and Modelability Indexes

Abstract

The reliability of a QSAR classification model depends on its capacity to achieve confident predictions of new compounds not considered in the building of the model. The results of this external validation process show the applicability domain (AD) of the QSAR model and, therefore, the robustness of the model to predict the property/activity of new molecules. In this paper we propose the use of the rivality and modelability indexes for the study of the characteristics of the datasets to be correctly modeled by a QSAR algorithm and to predict the reliability of the built model to prognosticate the property/activity of new molecules. The calculation of these indexes has a very low computational cost, not requiring the building of a model, thus being good tools for the analysis of the datasets in the first stages of the building of QSAR classification models. In our study, we have selected two benchmark datasets with similar number of molecules but with very different modelability and we have corroborated the capacity of the predictability of the rivality and modelability indexes regarding the classification models built using Support Vector Machine and Random Forest algorithms with 5-fold cross-validation and leave-one-out techniques. The results have shown the excellent ability of both indexes to predict outliers and the applicability domain of the QSAR classification models. In all cases, these values accurately predicted the statistic parameters of the QSAR models generated by the algorithms.

Predictive Capability of QSAR Models Based on the CompTox Zebrafish Embryo Assays: An Imbalanced Classification Problem

“…The concept of evaluating the predictability or modelability, essentially based on distance and similarity measures, has been developed in several studies [ 23 , 24 , 25 , 26 ]. The distance or similarity between molecules was estimated from the predictions of activities done by previously developed QSAR models.…”

“…Then, the modelability index for classification QSAR endpoint is defined as the ratio of compounds having the first nearest neighbor in the same class to the total number of compounds in the data set [ 24 ]. Later, also analogous modelability index based on the Euclidean distance measured between compounds in feature space and activity prediction by classification QSAR models was introduced for classification endpoints [ 25 ]. An alternative and conceptually simpler method for estimating modelability is the one used by Thomas et al [ 26 ] which is based only on the consideration of predictive capabilities of models comparing with the gain of the model over the level of random (chance) accuracy.…”

2D-QSAR and docking study of a series of coumarin derivatives as inhibitors of CDK (anticancer activity) with an application of the molecular docking method

“…The determination of ADs is therefore of great importance as it is explicitly requested in the validation processes put in place at the OECD level [ 39 ].…”

An In Silico Approach for Assessment of the Membrane Transporter Activities of Phenols: A Case Study Based on Computational Models of Transport Activity for the Transporter Bilitranslocase

“…The reliability of a QSAR model depends on its capacity to achieve confident predictions of new compounds not considered in the model development, which implies the necessity of considering the applicability domain (AD) of the model. Several AD analyses including the descriptor’s range, leverages, Euclidian distances, rivality and modelability indexes [ 66 , 67 , 68 ] are available. In this study we used two of AD approaches; maximal Euclidian distance of training set in NN-C (EDcrt = 6.16) and NN-D model (ED crt = 2.36), and leverage values in Q-D model (hat* = 0.4) ( Figure 4 ) [ 67 , 68 ].…”