In this paper we introduce a novel approach for classifier and feature selection in a multi-classifier system using Genetic Algorithm (GA). Specifically, we propose a 2-part structure for each chromosome in which the first part is encoding for classifier and the second part is encoding for feature. Our structure is simple in the implementation of the crossover as well as the mutation stage of GA. We also study 8 different fitness functions for our GA algorithm to explore the optimal fitness functions for our model. Experiments are conducted on both 14 UCI Machine Learning Repository and CLEF2009 medical image database to demonstrate the benefit of our model on reducing classification error rate.
Combining outputs from different classifiers to achieve high accuracy in classification task is one of the most active research areas in ensemble method. Although many state-of-art approaches have been introduced, no method is outstanding compared with the others on numerous data sources. With the aim of introducing an effective classification model, we propose a Gaussian Mixture Model (GMM) based method that combines outputs of base classifiers (called meta-data or Level1 data) resulted from Stacking Algorithm. We further apply Genetic Algorithm (GA) to that data as feature selection strategy to explore an optimal subset of Level1 in which our GMM-based approach can achieve high accuracy. Two methods are combined in a single framework called GAGMM. Experiments implemented on 21 UCI Machine Learning Repository data files and CLEF2009 medical image database demonstrate the advantage of our framework compared with other well-known combining algorithms such as Decision Template, Multiple Response Linear Regression (MLR), SCANN and fixed combining rules as well as GMM-based approaches on original data.
The paper introduces a novel 2-Stage model for multiclassifier system. Instead of gathering posterior probabilities resulted from base classifiers into Level1 data like in the original 2-Stage model, here we separate data in K Level1 matrices corresponding to the K base classifiers. These data matrices, in turn, are classified in sequence by a new classifier at the second stage to generate Level2 data. Next, Weight Matrix is proposed to combine Level2 data and predict label of observations in test set. Experimental results on CLEF2009 medical image database demonstrate the benefit of our model in comparison with several ensemble learning models.
Combining multiple classifiers to achieve better performances than any single classifier is one of the most important research areas in machine learning. In this paper, we focus on combining different classifiers to form an effective ensemble system. By introducing a novel framework operated on outputs of different classifiers, our aim is building a powerful model which is competitive with other well-known combining algorithms such as Decision Template, Multiple Response Linear Regression (MLR), SCANN and fixed combining rules. It is difference from the traditional approaches, here we use Gaussian Mixture Model (GMM) to model distribution of Level1 data and predict label of an interesting observation based on maximize of posterior probability through Bayes model. We also expand GMM-based approach in which before modeling distribution, Principle Component Analysis (PCA) method is applied to output of base classifiers to reduce its dimension; as a result, improve performance and availability of model based GMM. Experiments were evaluated on 21 UCI Machine Learning Repository demonstrate benefits of our framework compared with benchmarks.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.