Predictive Ensemble Modelling: Experimental Comparison of Boosting Implementation Methods

Abstract

-This paper presents the empirical comparison of boosting implementation by reweighting and resampling methods. The goal of this paper is to determine which of the two methods performs better. In the study, we used four algorithms namely: Decision Stump, Neural Network, Random Forest and Support Vector Machine as base classifiers and AdaBoost as a technique to develop various ensemble models. We applied 10-fold cross validation method in measuring and evaluating the performance metrics of the models. The results show that in both methods the average of the correctly classified and incorrectly classified are relatively the same. However, average values of the RMSE in both methods are insignificantly different. The results further show that the two methods are independent of the datasets and the base classier used. Additionally, we found that the complexity of the chosen ensemble technique and boosting method does not necessarily lead to better performance.

Comparative Analysis of Machine Learning Models for Auto Insurance Fraud Detection

“…In Bagging, the combination of weak classifiers is used to improve the classification accuracy. [13] The idea of bagging is to fit several independent models and "average" their predictions in order to obtain a model with a lower variance. [5] Bagging can be applied to any type of machine learning algorithm, but it is particularly effective with decision trees, which are known to have high variance.…”