A method for calibration and validation subset partitioning

Galvão, Roberto Kawakami Harrop; Araújo, Mário César Ugulino de; José, Gledson Emídio; Pontes, Márcio José Coelho; Silva, Edvan Cirino; Saldanha, Teresa Cristina Bezerra

doi:10.1016/j.talanta.2005.03.025

Cited by 757 publications

(332 citation statements)

References 12 publications

Supporting

Mentioning

301

Contrasting

Unclassified

Order By: Relevance

“…Because of its consideration of the variability in both x and y-information, the SPXY was cited as a method that can cover the multidimensional space more effectively in comparison with the randomly sampling (RS) or partitioning schemes based only on x-or y-space alone (such as the Kennard-Stone (KS) algorithm or concentration gradient sampling) [23]. Consequently, this method can be potentially used to improve the performance of the predictive models.…”

Section: Discussionmentioning

confidence: 99%

Non-destructive Measurement of Sugar Content in Chestnuts Using Near-Infrared Spectroscopy

Liu

et al. 2011

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

Abstract. The chestnut (Castanea) is an important fruit in Europe and Asia. As a highly variable fruit, its quality is graded according to nutrition components, especially according to the sugar content, which are traditionally measured by using chemical methods. However, the traditional methods are time-consuming, laborious, and expensive. Here, we analyzed the sugar content of intact and peeled chestnuts by near-infrared spectroscopy. The spectra of intact and peeled chestnut samples were collected in the wavelength range from 833 nm to 2500 nm. The Sample Set Partitioning based on joint X-Y distances was used when the calibration and validation subsets were partitioned. The predictive models for intact and peeled chestnut samples respectively, were developed using partial least squares (PLS) regression based on the original spectra and the spectra derived from different pretreatments. The PLS models developed from the spectra of peeled samples gave accurate predictions. The correlation coefficient (R 2 ) of the optimized model for calibration set and validation set were 0.90 and 0.86. Although the models established on the spectra of intact samples did not perform excellently, they were still qualified to measure sugar content of the chestnut kernel. The correlation coefficient (R 2 ) of optimized model for calibration set and validation set were 0.89 and 0.59. These results suggested that NIR spectroscopy could be used as a fast and accurate alternative method for the nondestructive evaluation of sugar content in chestnuts during orchard and postharvest processes.

show abstract

Section: Discussionmentioning

confidence: 99%

Non-destructive Measurement of Sugar Content in Chestnuts Using Near-Infrared Spectroscopy

Liu

et al. 2011

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

show abstract

“…When dividing the limited dataset into training and testing data, the training and testing data should be representative of the entire dataset. For this purpose, the SPXY algorithm [43], developed from the classic Kennard-Stone algorithm [44], is employed to extract the representative training data, while the rest are used for testing. The actual partition based on SPXY is dependent on the initially selected training sample.…”

Section: Implemental Detailsmentioning

confidence: 99%

“…It is crucial to ensure that the training/testing data are representative of the whole dataset in order to conduct reliable and meaningful evaluation. This objective can be implemented by the application of the SPXY algorithm (sample set partitioning based on joint x-y distances) [43], which is a development of the classic Kennard-Stone algorithm [44]. The predictive accuracy, quantified by root mean square error of prediction (RMSEP), is used to evaluate the performance of the investigated regression techniques.…”

Section: Introductionmentioning

confidence: 99%

Bagging for robust non-linear multivariate calibration of spectroscopy

Wang

Chen

Lau

2011

Chemometrics and Intelligent Laboratory Systems

View full text Add to dashboard Cite

“…They consisted of 60 spectra of the gasoline samples. These data were partitioned to training and test sets with size of 40 and 20, respectively, by a method termed as SPXY (sample set partitioning based on joint x-y distances) which is described in Reference [33]. The corresponding spectra are shown in Figure 4. 3.5.…”

Section: Data Set 3 (Nir Data For Gasoline Samples) [31]mentioning

confidence: 99%

Ant colony optimisation: a powerful tool for wavelength selection

Shamsipur

Zare-Shahabadi

Hemmateenejad

et al. 2006

Journal of Chemometrics

View full text Add to dashboard Cite

Ant colony optimisation (ACO) is a meta-heuristic algorithm, which is derived from the observation of real ants. Real ant colonies are distributed system that, in spite of the simplicity of their individuals, present a highly structured social organisation and can accomplish complex tasks. They always find a short path between the nest and a food source. ACO is based on local message exchange via the deposition of pheromone trails. It is in fact a population-based approach using positive feedback as well as greedy search. Wavelength selection is a strategy used for improving the quality of calibration methods. As a first report, this work indicated that the ACO possesses a great ability to find best subsets of wavelengths, at a short period of time with small PRESS values, via accumulation of information in the form of pheromone trails deposited on each wavelength. Theory of ACO is described and, to carry out the wavelength selection, a fitness function is defined. The ACO parameters are configured with a 3-levels full factorial design. The high ability of ACO in wavelength selection process was demonstrated by examining four different NIR and UV/Vis data sets via various ACO algorithms, including ACO-ILS, ACO-CLS and ACO-PLS. The results showed that, with the same fitness function, ACO-ILS algorithm suffers from some overfitting problem. This problem was overcome by constraining the algorithm to choose limited number of wavelengths, the corresponding algorithm called as ACO-ILS(limited). The results obtained by these algorithms clearly revealed the improved predictive ability of ACO in wavelength selection over the existing full-spectrum models.

show abstract

A method for calibration and validation subset partitioning

Cited by 757 publications

References 12 publications

Non-destructive Measurement of Sugar Content in Chestnuts Using Near-Infrared Spectroscopy

Non-destructive Measurement of Sugar Content in Chestnuts Using Near-Infrared Spectroscopy

Bagging for robust non-linear multivariate calibration of spectroscopy

Ant colony optimisation: a powerful tool for wavelength selection

Contact Info

Product

Resources

About