Performance of Machine Learning Algorithms with Different K Values in K-fold CrossValidation

Abstract

The numerical value of k in a k-fold cross-validation training technique of machine learning predictive models is an essential element that impacts the model’s performance. A right choice of k results in better accuracy, while a poorly chosen value for k might affect the model’s performance. In literature, the most commonly used values of k are five (5) or ten (10), as these two values are believed to give test error rate estimates that suffer neither from extremely high bias nor very high variance. However, there is no formal rule. To the best of our knowledge, few experimental studies attempted to investigate the effect of diverse k values in training different machine learning models. This paper empirically analyses the prevalence and effect of distinct k values (3, 5, 7, 10, 15 and 20) on the validation performance of four well-known machine learning algorithms (Gradient Boosting Machine (GBM), Logistic Regression (LR), Decision Tree (DT) and K-Nearest Neighbours (KNN)). It was observed that the value of k and model validation performance differ from one machine-learning algorithm to another for the same classification task. However, our empirical suggest that k = 7 offers a slight increase in validations accuracy and area under the curve measure with lesser computational complexity than k = 10 across most MLA. We discuss in detail the study outcomes and outline some guidelines for beginners in the machine learning field in selecting the best k value and machine learning algorithm for a given task.

Mapping the impacts of multiple stressors on the decline in kelps along the coast of Victoria, Australia

“…We also used a 10 k‐fold cross‐validation method to further reduce instances of overfitting by testing the predictive accuracy of the training dataset (Leathwick et al, 2006). The 10 k‐fold cross‐validation works by splitting the training data into 10 equal parts and then leaving one part out each time the model is run to test the predictive performance, which can indicate selection bias or overfitting in the model (Nti et al, 2021). The BRT models were run using R statistical software (Team, 2014) and the “gbm” package (Ridgeway, 2007).…”

Machine Learning Approach for Predictive Maintenance of the Electrical Submersible Pumps (ESPs)

“…When performing cross validation, It is typically a standard that the chosen number of folds is equal to 10 ( k = 10). 45 …”

“…The hypothesis resulting from such operation would be the final answer. When performing cross validation, It is typically a standard that the chosen number of folds is equal to 10 ( k = 10) …”

“…When performing cross validation, It is typically a standard that the chosen number of folds is equal to 10 (k = 10). 45 Hyperparameter tuning is considered one of the important steps while creating any data-driven model to get the best results from the deployed algorithm. Regarding the XGBoost algorithm, hyperparameters are divided into three categories.…”

Klasifikasi Dialek Bahasa Jawa Menggunakan Metode Naives Bayes

“…Sebagian besar penggungaan fold=10 dapat memberikan hasil yang baik tetapi terkadang di beberapa kasus fold=5 sudah cukup memadai [32]. Pada penelitian yang dilakukan Nti [33], nilai fold=7 dapat memberikan peningkatan dalam akurasi validasi. Selain itu penelitian yang dilakukan Tempola [34] dibagi kedalam 3 fold disetiap metode klasifikasi diperoleh perbandingan akurasi sistem rata-rata tertinggi.…”