Kennard-Stone method outperforms the Random Sampling in the selection of calibration samples in SNPs and NIR data

Abstract

Splitting the whole dataset into training and testing subsets is a crucial part of optimizing models. This study evaluated the influence of the choice of the training subset in the construction of predictive models, as well as on their validation. For this purpose we assessed the Kennard-Stone (KS) and the Random Sampling (RS) methods in near-infrared spectroscopy data (NIR) and marker data SNPs (Single Nucleotide Polymorphisms). It is worth noting that in SNPs data, there is no knowledge of reports in the literature regarding the use of the KS method. For the construction and validation of the models, the partial least squares (PLS) estimation method and the Bayesian Lasso (BLASSO) proved to be more efficient for NIR data and for marker data SNPs, respectively. The evaluation of the predictive capacity of the models obtained after the data partition occurred through the correlation between the predicted and the observed values, and the corresponding square root of the mean squared error of prediction. For both datasets, results indicated that the results from KS and RS methods differ statistically from each other by the F test (P-value < 0.01). The KS method showed to be more efficient than RS in practically all repetitions. Also, KS method has the advantage of being easy and fast to be applied and also to select the same samples, which provides excellent benefits in the following analyses.

Quantitative analysis of heavy metals in soilviahierarchical deep neural networks with X-ray fluorescence spectroscopy

“…This ensures that there is no data leakage during the calculation process, while also ensuring the consistency and reproducibility of the dataset. 41–43 Based on the full dataset, 36 samples were assigned to the training set, 12 samples to the validation set, and the remaining 11 samples to the test set.…”

“…This ensures that there is no data leakage during the calculation process, while also ensuring the consistency and reproducibility of the dataset. [41][42][43] Based on the full dataset, 36 samples were assigned to the training set, 12 samples to the validation set, and the remaining 11 samples to the test set. The split data were input to different elemental concentration prediction models, including HDNN, DNN, BPNN, and PLSR methods, to prove the superiority of our proposed HDNN method.…”

A Novel Tran_NAS Method for the Identification of Fe- and Mg-Deficient Pear Leaves from N- and P-Deficient Pear Leaf Data

“…Prior to modeling, the dataset was divided into a training set and a test set according to the Kennard–Stone algorithm 41 to show the classification ability of the model. To assess the relative robustness of the various preprocessing methods, modeling is performed next, and the modeling results are presented and analyzed in Section 3.2 .…”

A deep spectral prediction network to quantitatively determine heavy metal elements in soil by X-ray fluorescence

“…The Kennard-stone algorithm was utilized to select 80% of them as the training set and the remaining 20% as the prediction set. 35 Thus, the training set has 216 sets of samples and the prediction set contains 54 sets of samples. The input form of the soil XRF spectral data is 800 × 1.…”