Optimal Classifier Fusion in a Non-Bayesian Probabilistic Framework

Abstract

The combination of the output of classifiers has been one of the strategies used to improve classification rates in general purpose classification systems. Some of the most common approaches can be explained using the Bayes' formula. In this paper, we tackle the problem of the combination of classifiers using a non-Bayesian probabilistic framework. This approach permits us to derive two linear combination rules that minimize misclassification rates under some constraints on the distribution of classifiers. In order to show the validity of this approach we have compared it with other popular combination rules from a theoretical viewpoint using a synthetic data set, and experimentally using two standard databases: the MNIST handwritten digit database and the GREC symbol database. Results on the synthetic data set show the validity of the theoretical approach. Indeed, results on real data show that the proposed methods outperform other common combination schemes.

The generalization of the R-transform for invariant pattern representation

“…symbol recognition [13], activity recognition [14,18], and orientation estimation [7]. This paper provides a generic view on the R-transform and the R-signature while maintaining their beneficial properties, leading to two main theoretical contributions.…”

Reduced Analytical Dependency Modeling for Classifier Fusion

“…Max, Min, Median and Majority Vote, can be derived. Besides these combination rules developed under Bayesian framework [3], Terrades et al [7] tackled the classifier combination problem using a non-Bayesian probabilistic framework. Under the assumptions that classifiers can be combined linearly and the scores follow independent normal distribution, the independent normal (IN) combination rule was derived [7].…”

“…For example, in [5], Parzen window density estimation was used to estimate the joint density of posterior probabilities by a selected set of classifiers. Since it needs numerous data to ensure that estimation of the joint distribution is accurate [11], Terrades et al [7] proposed to combine classifiers by a linear model under normal distribution assumption. When features are not conditionally independent, the covariance matrix in the normal distribution is not diagonal.…”

“…To solve this problem, many algorithms have been developed to extract local or global discriminative features such as Local Binary Patterns [1], Laplacianfaces [2], etc. Instead of extracting a high discriminative feature, classifier fusion has been proposed and the results are encouraging [3] [4] [5] [6] [7] [8] [9] [10]. While many classifier combination techniques [3] have been studied and developed in the last decade, it is a general assumption that classification scores are conditionally independent distributed.…”

“…However, when the number of classifiers is large, it needs numerous data to accurately estimate the joint density [11]. On the other hand, Terrades et al [7] proposed to combine classifiers in a non-Bayesian framework by linear combination. Under dependent normal assumption (DN), they formulated the classifier combination problem into a constraint quadratic programming problem.…”