Vote counting measures for ensemble classifiers

IEEE Trans. Neural Netw. Learning Syst.

Zor

Camgöz

2019

Abstract-A spectral analysis of a Boolean function is proposed for approximating the decision boundary of an ensemble of classifiers, and an intuitive explanation of computing Walsh coefficients for the functional approximation is provided. It is shown that the difference between first and third order coefficient approximation is a good indicator of optimal base classifier complexity. When combining Neural Networks, experimental results on a variety of artificial and real two-class problems demonstrate under what circumstances ensemble performance can be improved. For tuned base classifiers, first order coefficients provide performance similar to majority vote. However, for weak/fast base classifiers, higher order coefficient approximation may give better performance. It is also shown that higher order coefficient approximation is superior to the Adaboost logarithmic weighting rule when boosting weak Decision Tree base classifiers.

Section: Introductionmentioning

confidence: 99%

Approximation of Ensemble Boundary Using Spectral Coefficients

IEEE Trans. Neural Netw. Learning Syst.

Zor

Camgöz

2019

“…This mapping may be analysed using Walsh spectral coefficients. First order Walsh coefficients were shown to provide a measure of class separability for selecting optimal base classifiers in [2], in which it is also shown that this does not imply optimality of the ensemble. In contrast, in [3] it was shown that second order Walsh coefficients may be used to determine optimal ensemble performance.…”

Section: Introductionmentioning

confidence: 99%

“…In contrast, in [3] it was shown that second order Walsh coefficients may be used to determine optimal ensemble performance. The motivation for using Walsh coefficients in ensemble design is fully explored in [4] and [2]. For further understanding of the meaning and applications of Walsh coefficients see [5] and [6].…”

Section: Introductionmentioning

confidence: 99%

Low Training Strength High Capacity Classifiers for Accurate Ensembles Using Walsh Coefficients

Lecture Notes in Computer Science

Zor

2012

Self Cite

Abstract. If a binary decision is taken for each classifier in an ensemble, training patterns may be represented as binary vectors. For a two-class supervised learning problem this leads to a partially specified Boolean function that may be analysed in terms of spectral coefficients. In this paper it is shown that a vote which is weighted by the coefficients enables a fast ensemble classifier that achieves performance close to Bayes rate. Experimental evidence shows that effective classifier performance may be achieved with one epoch of training of an MLP using Levenberg-Marquardt with 64 hidden nodes.

“…The problem with applying classical class separability measures to the binary mapping associated with MCS (equation ( (1))) is that the implicit Gaussian assumption is not appropriate [11]. In [4], [12] a class separability measure is proposed for MCS that is based on a binary feature representation, in which each pattern is represented by its binary ensemble classifier decisions. It is restricted to two-class problems and results in a binary-to-binary mapping.…”

Section: Diversity/accuracy and Mcsmentioning

confidence: 99%

“…The measure is based on a spectral representation that was first proposed for two-class problems in [2], and later developed in the context of Multiple Classifier Systems in [3]. It was shown for two-class problems in [4] that over-fitting of the training set could be detected by observing the separability measure as it varies with base classifier complexity. Since realistic learning problems are in general ill-posed [5], it is known that any attempt to automate the learning task must make some assumptions.…”

Section: Introductionmentioning

confidence: 99%

Ensemble MLP Classifier Design

Studies in Computational Intelligence

Self Cite

Abstract. Multi-layer perceptrons (MLP) make powerful classifiers that may provide superior performance compared with other classifiers, but are often criticized for the number of free parameters. Most commonly, parameters are set with the help of either a validation set or crossvalidation techniques, but there is no guarantee that a pseudo-test set is representative. Further difficulties with MLPs include long training times and local minima. In this chapter, an ensemble of MLP classifiers is proposed to solve these problems. Parameter selection for optimal performance is performed using measures that correlate well with generalisation error.