HDEC: A Heterogeneous Dynamic Ensemble Classifier for Binary Datasets

Abstract

In recent years, ensemble classification methods have been widely investigated in both industry and literature in the field of machine learning and artificial intelligence. The main advantage of this approach is to benefit from a set of classifiers instead of using a single classifier with the aim of improving the prediction performance, such as accuracy. Selecting the base classifiers and the method for combining them are the most challenging issues in the ensemble classifiers. In this paper, we propose a heterogeneous dynamic ensemble classifier (HDEC) which uses multiple classification algorithms. The main advantage of using heterogeneous algorithms is increasing the diversity among the base classifiers as it is a key point for an ensemble system to be successful. In this method, we first train many classifiers with the original data. Then, they are separated based on their strength in recognizing either positive or negative instances. For doing this, we consider the true positive rate and true negative rate, respectively. In the next step, the classifiers are categorized into two groups according to their efficiency in the mentioned measures. Finally, the outputs of the two groups are compared with each other to generate the final prediction. For evaluating the proposed approach, it has been applied to 12 datasets from the UCI and LIBSVM repositories and calculated two popular prediction performance metrics, including accuracy and geometric mean. The experimental results show the superiority of the proposed approach in comparison to other state-of-the-art methods.

An approach for improved students’ performance prediction using homogeneous and heterogeneous ensemble methods

“…This section briefly introduces an overview of ensemble learning methods, related works, and prediction approaches. The main goal of an ensemble classifier is to take advantage of the benefits of multiple classifiers and combine their outputs so that the predictive accuracy of the model improves [23]. The individual classifiers in an ensemble system are referred to as base classifiers.…”

“…Ostvar and Moghadam [23] introduced a heterogeneous dynamic ensemble classifier (HDEC), which used multiple classification algorithms and tested it on 12 standard datasets from the University of California Irvine (UCI) repository. They compared the performance of their proposed method to three cutting-edge ensemble approaches namely bagging, boosting, and stack generalization.…”

“…No single classification algorithm is considered optimal for all cases, and only by combining various single classifiers can classification performance be improved [20], [21]. The main goal of ensembles is to improve generalization and diversity among the models to deal with dataset variance, and only HTE can achieve this better because it uses a diverse set of base classifiers [14], [22], [23]. Experimental results of various studies demonstrated the effectiveness of using HTE with significantly improved performance when compared to HE [15], [24]- [26].…”

“…In bagging, each base classifier was trained with a bootstrapped replica of the training dataset [23]. Then, the final decision was generated by applying a majority voting on each base classifier's decision.…”

Ensemble learning based on relative accuracy approach and diversity teams

“…Ensemble learning has many approaches and many works looked at considering many aspects. Some works focused on the types of trainers, either homogenous [3], [4] or heterogeneous [5], [6]. Some works looked at the purpose of using ensemble learning either for classifying [7]- [26], clustering [27]- [34], regression [35]- [37], or streaming [38]- [42].…”