On Selection of Training and Test Sets for the Development of Predictive QSAR models

Abstract

The development of predictive QSAR models depends not only on the statistical method but also on the algorithm used for the selection of training and test sets. Here, we describe the validation of QSAR models for three data sets with different sizes (n ¼ 35, 56 and 87) based on random division, sorted biological activity data and K-means clusters for the factor scores of the original variable matrix along with/without biological activity values. When the training and test sets were generated by random division or by the activity-range algorithm, predictive models were not obtained in most of the cases. In case of random division of the data sets into training and test sets, there is no correlation between internal and external validation statistics. However, good external validation statistics were obtained when training and test sets were selected based on K-means clusters of factor scores of the descriptor space along with/without the biological activity values. So, the selection of training and test sets should be based on the proximity of the representative points of the test set to representative points of the training set in the multidimensional descriptor space. The concept of closeness is based on the general assumption underlying all QSAR theories: similar compounds have similar activities. Thus, if one wishes to validate a QSAR model, the points of the test set must be close to the points of the training set in the multidimensional descriptor space. Based on the results of several methods for the division of the training and test sets, we propose that K-means-clusterbased division of training and prediction sets can be used as a reliable method of division of data set into training and test sets for developing predictive QSAR models.

Development of QSAR models using artificial neural network analysis for risk assessment of repeated-dose, reproductive, and developmental toxicities of cosmetic ingredients

“…Since the performance of a QSAR model depends on the numerical values of the NOELs used in training the model (Leonard and Roy, 2006), training of a QSAR model for assessing the toxicity of cosmetic ingredients should ideally be done with the aid of datasets of potential or actual cosmetic ingredients (e.g. chemicals from the International Nomenclature of Cosmetic Ingredients (INCI) list).…”

Predictive QSAR modeling of CCR5 antagonist piperidine derivatives using chemometric tools

“…So, the selection of the training set is significantly important in QSAR analysis. Predictive potential of a model on the new data set is influenced by the similarity of chemical nature between training set and test set [28][29][30]. The test set molecules will be predicted well when these molecules are very similar to the training set compounds.…”

3D-QSAR studies of triazolopyrimidine derivatives of Plasmodium falciparum dihydroorotate dehydrogenase inhibitors using a combination of molecular dynamics, docking, and genetic algorithm-based methods

“…It has been indicated that to achieve the optimal model, the selection of training and test sets should be based on some rational algorithms; otherwise, poor predictive ability of QSAR models may be obtained [7]. Therefore, it is also an important step to select the group of molecules that represent the most critical structural and physicochemical features associated with activity.…”