The Impact of Selecting a Validation Method in Machine Learning on Predicting Basketball Game Outcomes

Abstract

Interest in sports predictions as well as the public availability of large amounts of structured and unstructured data are increasing every day. As sporting events are not completely independent events, but characterized by the influence of the human factor, the adequate selection of the analysis process is very important. In this paper, seven different classification machine learning algorithms are used and validated with two validation methods: Train&Test and cross-validation. Validation methods were analyzed and critically reviewed. The obtained results are analyzed and compared. Analyzing the results of the used machine learning algorithms, the best average prediction results were obtained by using the nearest neighbors algorithm and the worst prediction results were obtained by using decision trees. The cross-validation method obtained better results than the Train&Test validation method. The prediction results of the Train&Test validation method by using disjoint datasets and up-to-date data were also compared. Better results were obtained by using up-to-date data. In addition, directions for future research are also explained. Symmetry 2020, 12, 431 2 of 15The aim of this paper is, through the comparison of the classification machine learning algorithms in predicting basketball game outcomes, to define which algorithm, validation method, and data preparation method produces better prediction results.This paper demonstrates what impact the different validation methods have on the prediction accuracy when using different ML algorithms. Moreover, the impact of selecting a validation method on the prediction results when applying ML to the disjoint datasets or the up-to-date data is revealed, thereby enabling the formation of recommendations for the most appropriate combination of the ML algorithm and validation method, depending on the available datasets.After this introduction and the overview of sport outcome-related researches, the second chapter provides the basic information about classification machine learning algorithms and the validation methods applied in this research. The third chapter describes the data acquisition and data preparation procedures. The research results are presented and discussed in the fourth chapter, and the conclusions are given at the end of paper. Related Literature ReviewThe most common algorithm in predicting outcomes in sports are neural networks coupled with the Train&Test validation method. The authors of [1] used a variety of neural networks and Train&Test validation for predicting game outcomes in the National Basketball Association (NBA) league, with the best results of more than 70%. In [2], the authors used 37 algorithms in the Waikato Environment for Knowledge Analysis (WEKA) and Train&Test validation method. The result with the best yield was 72.8%, showing that the best classifiers have 5% better precision than the referent classifier, which favors the team with the better rating. The authors of [3] used logistic regression, Naïve Bayes, Support Vector Machine (S...

Who Will Score? A Machine Learning Approach to Supporting Football Team Building and Transfers

“…In [ 6 ], the validation step was investigated. The authors claimed that there are two possible validation methods for predicting the results of basketball matches, namely train & test and cross-validation.…”

Hybrid Basketball Game Outcome Prediction Model by Integrating Data Mining Methods for the National Basketball Association

“…There are many different prediction techniques have been applied in different areas, including civil engineering [ 40 ], industrial engineering [ 41 , 42 ], healthcare [ 43 , 44 ], safety [ 45 ] and data mining [ 46 , 47 ]. Sports outcomes prediction has attracted attention in different sports [ 1 , 2 , 3 , 4 , 5 , 6 ]. For example, the author of [ 7 ] built a simple, weighted and penalized regression model using the match-up, date and final score records to predict baseball, basketball, American football and hockey outcomes.…”

“…Sports outcomes prediction has attracted attention in different sports [ 1 , 2 , 3 , 4 , 5 , 6 ]. However, most of the existing studies in this regard are focused on basketball, in particular, on the National Basketball Association (NBA) games, since the NBA is the most popular basketball league in the world [ 4 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 ]. Most existing research related to basketball games is aimed at predicting the win/lose (W/L) outcomes of a game [ 3 , 4 , 6 , 8 , 9 , 10 , 12 , 15 ].…”

“…However, most of the existing studies in this regard are focused on basketball, in particular, on the National Basketball Association (NBA) games, since the NBA is the most popular basketball league in the world [ 4 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 ]. Most existing research related to basketball games is aimed at predicting the win/lose (W/L) outcomes of a game [ 3 , 4 , 6 , 8 , 9 , 10 , 12 , 15 ]. Only a few studies have focused on the prediction of basketball game scores [ 11 , 12 , 13 , 14 , 15 ].…”

“…The five well-known data mining methods, including multivariate adaptive regression splines (MARS), k-nearest neighbors (KNN), extreme learning machine (ELM), eXtreme gradient boosting (XGBoost), and stochastic gradient boosting (SGB), are used in this study for building an NBA game score prediction model, as they have been widely used in various applications such as public health [ 17 , 18 ], finance [ 19 , 20 ] and civil engineering [ 21 , 22 ]. Moreover, the five methods are also successfully applied in the sports outcomes prediction research [ 6 , 23 , 24 , 25 , 26 ].…”