Combining Classifiers Based on Gaussian Mixture Model Approach to Ensemble Data

Abstract

Combining multiple classifiers to achieve better performances than any single classifier is one of the most important research areas in machine learning. In this paper, we focus on combining different classifiers to form an effective ensemble system. By introducing a novel framework operated on outputs of different classifiers, our aim is building a powerful model which is competitive with other well-known combining algorithms such as Decision Template, Multiple Response Linear Regression (MLR), SCANN and fixed combining rules. It is difference from the traditional approaches, here we use Gaussian Mixture Model (GMM) to model distribution of Level1 data and predict label of an interesting observation based on maximize of posterior probability through Bayes model. We also expand GMM-based approach in which before modeling distribution, Principle Component Analysis (PCA) method is applied to output of base classifiers to reduce its dimension; as a result, improve performance and availability of model based GMM. Experiments were evaluated on 21 UCI Machine Learning Repository demonstrate benefits of our framework compared with benchmarks.