Impact assessment of the rational selection of training and test sets on the predictive ability of QSAR models

Abstract

This study performed an analysis of the influence of the training and test set rational selection on the quality and predictively of the quantitative structure-activity relationship (QSAR) model. The study was carried out on three different datasets of Influenza Neuraminidase (H1N1) inhibitors. The three datasets were divided into training and test sets using three rational selection methods: based on k-means, Kennard-Stone algorithm and Activity and the results were compared with Random selection. Then, a total of 31,490 mathematical models were developed and those models that presented a determination coefficient higher than: r > 0.8, r > 0.7, r > 0.5 and minimum standard deviation (SD) and minimum root-mean square error (RMS) were selected. The selected models were validated using the internal leave-one-out method and the predictive capacity was evaluated by the external test set. The results indicate that random selection could lead to erroneous results. In return, a rational selection allows for obtaining more reliable conclusions. The QSAR models with major predictive power were found using the k-means algorithm and selection by activity.

Assessing the calibration in toxicological in vitro models with conformal prediction

“…Several groups investigated random vs. rational selection of optimal test/training sets, e.g. using cluster-or activity-based splits, with the goal of better reflecting the true predictive power of established models [10][11][12][13][14]. Martin et al [11] showed that rational selection of training and test sets-compared to random splitsgenerated better statistical results on the (internal) test sets.…”

Polysaccharide determination and habitat classification for fresh Dendrobiums with hyperspectral imagery and modified RBFNN

“…Then the remained samples were divided into calibration set and prediction set at a ratio of 2 : 1 according to the K–S algorithm. 28 The PLSR models were first built by the calibration set and tested by the prediction set to testify the feasibility of spectral detection on Dendrobiums . For both MN and DP, the R P 2 values of PLSR models are greater than 0.84 ( Table 1 ).…”

A Comprehensive QSAR Study on Antileishmanial and Antitrypanosomal Cinnamate Ester Analogues

“…The set of compounds was independently divided twice to yield two different training and test sets by random selection. Additionally, a third division into training and test set was obtained based on maximum dissimilarity using the Kennard-Stone algorithm [29,30,31,32]. The ratio training/test set was kept constant in all three sets ( N = 26 for training and N = 8 for test set).…”