The importance of choosing a proper validation strategy in predictive models. A tutorial with real examples

Lopez, Eneko; Etxebarria-Elezgarai, Jaione; Amigo, José Manuel; Seifert, Andreas

doi:10.1016/j.aca.2023.341532

Cited by 13 publications

(3 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cross-validation helps assess the performance and generalization ability of the models by minimizing the risk of overfitting or underfitting. By evaluating the model on unseen data in each iteration, cross-validation can provide a reliable estimate of how likely the model is to perform well on new, unseen samples. , …”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Power of Light: Raman Spectroscopy and Machine Learning for the Detection of Lung Cancer

Hano,

Lawrie,

Suarez

et al. 2024

ACS Omega

Self Cite

View full text Add to dashboard Cite

Lung cancer is the leading cause of cancer-related deaths worldwide, emphasizing the urgent need for reliable and efficient diagnostic methods. Conventional approaches often involve invasive procedures and can be time-consuming and costly, thereby delaying the effective treatment. The current study explores the potential of Raman spectroscopy, as a promising noninvasive technique, by analyzing human blood plasma samples from lung cancer patients and healthy controls. In a benchmark study, 16 machine learning models were evaluated by employing four strategies: the combination of dimensionality reduction with classifiers; application of feature selection prior to classification; stand-alone classifiers; and a unified predictive model. The models showed different performances due to the inherent complexity of the data, achieving accuracies from 0.77 to 0.85 and areas under the curve for receiver operating characteristics from 0.85 to 0.94. Hybrid methods incorporating dimensionality reduction and feature selection algorithms present the highest figures of merit. Nevertheless, all machine learning models deliver creditable scores and demonstrate that Raman spectroscopy represents a powerful method for future in vitro diagnostics of lung cancer.

show abstract

Section: Resultsmentioning

confidence: 99%

“…By evaluating the model on unseen data in each iteration, cross-validation can provide a reliable estimate of how likely the model is to perform well on new, unseen samples. 25 , 26 …”

Section: Resultsmentioning

confidence: 99%

Power of Light: Raman Spectroscopy and Machine Learning for the Detection of Lung Cancer

Hano,

Lawrie,

Suarez

et al. 2024

ACS Omega

Self Cite

View full text Add to dashboard Cite

show abstract

“…To further validate the efficacy of the PLS-DA approach based on a Fast Fourier transform, a permutation test with external cross-validation was implemented. Permutation test, as a ‘random algorithm’ based on the probabilistic notion, aimed to assess the importance of the approach's outcomes by randomly permuting the response matrix (Y) ( de Andrade, de Gois, Xavier, & Luna, 2020 ) and then the approach was rebuilt to utilize the same modeling settings to compute the probability outcomes that occur by chance ( Lopez, Etxebarria-Elezgarai, Amigo, & Seifert, 2023 ). It serves as a powerful tool to evaluate the validity of regression methodologies ( Ballabio & Consonni, 2013 ), where the Q 2 score is used to gauge the predictive capability, while R 2 is employed to measure the explanatory power.…”

Section: Resultsmentioning

confidence: 99%

The utilization of a data fusion approach to investigate fingerprint profiles of dark tea from China's different altitudes

Wang,

Han,

Zhang

et al. 2024

Food Chemistry: X

View full text Add to dashboard Cite

Near-infrared spectroscopy and multivariate analysis as real-time monitoring strategy of TEMPO-mediated oxidation of cellulose fibers from different feedstocks

Mazega,

Fortuny,

Signori-Iamin

et al. 2024

Cellulose

View full text Add to dashboard Cite

The regioselective oxidation of the primary hydroxyl groups of cellulose, usually mediated by the (2,2,6,6-tetramethylpiperidin-1-yl)oxyl radical (TEMPO), is highly popular in the scientific literature. However, the lack of efficient monitoring techniques imposes a severe limitation to its upscaling. This work involves a portable, user-friendly near-infrared spectroscopy device, optimized preprocessing techniques, and multivariate calibration to quickly estimate the carboxyl group content of modified cellulose (i.e., the extent of the oxidation). For that, bleached pulps from eucalyptus, pine, hemp, and sisal were submitted to TEMPO-mediated oxidation, varying the dosage of spent oxidizer (NaClO) and thus attaining samples of different values of carboxyl group content. These values were related to near-infrared spectra (908–1676 nm) by Partial Least Squares regression, yielding cross-validation coefficients (RCV2) above 0.97 for wood pulps, 0.95 for sisal pulp, and 0.91 for hemp pulp. Based on the residual prediction deviation, the model for each pulp was found to show good predictability. Nonetheless, the overall regression model, comprising the four different materials, was unreliable. In light of this, spectra were submitted to principal components analysis (PCA), hinting that pulps could be classified in terms of their hemicellulose to cellulose ratio. Considering all the statistical parameters, the overall proposal presented here begins with a PCA—Linear Discriminant Analysis model to classify the sample by its fiber type, subsequently selecting a specific regression model for that class. Overall, the presented models in this work allow the determination of the extent of oxidation of different cellulosic feedstocks, expressed as carboxyl content, in a fast and simple approach using a benchtop near-infrared equipment.

show abstract

The importance of choosing a proper validation strategy in predictive models. A tutorial with real examples

Cited by 13 publications

References 26 publications

Power of Light: Raman Spectroscopy and Machine Learning for the Detection of Lung Cancer

Power of Light: Raman Spectroscopy and Machine Learning for the Detection of Lung Cancer

The utilization of a data fusion approach to investigate fingerprint profiles of dark tea from China's different altitudes

Near-infrared spectroscopy and multivariate analysis as real-time monitoring strategy of TEMPO-mediated oxidation of cellulose fibers from different feedstocks

Contact Info

Product

Resources

About