A Feature-Based Serial Approach to Classifier Combination

Abstract

Originality and ContributionThe main motivation behind existing methods of classifier combination is improvement of classification accuracy. In this paper we are concerned with a different problem, which is improvement of computational efficiency. Reducing computational effort is extremely important in data-intensive applications of pattern recognition such as clickstream analysis on the web. The classification rate of "lazy" learning methods, like k-NN Classifier, is especially sensitive to the dimensionality of the training set. The computational complexity of a k-NN Classifier depends on two main factors: the number of distance computations (proportional to the number of training examples considered for finding the nearest neighbor of a new instance) and the effort associated with calculating a distance to each training example (proportional to the number of features).Several algorithms have been suggested to reduce the number of distance computations. To the best of our knowledge, this work presents the first attempt to reduce the computational complexity To sum-up, this work is original in at least the following aspects: it uses a feature-based approach to reducing the computational burden of an instance-based classifier; it proposes a serial combination of two k-NN classifiers; and it presents a new feature selection method ("IFN + Relief"). Evaluation results demonstrate the broad practical implications of the proposed methodology.

Test Set Generation and Reduction With Artificial Neural Networks

“…The method does not require the knowledge of either the tested code or the requirements, except for a list of system inputs and outputs along with their respective data types. The superiority of IFN for the feature selection task has been shown in [23]. This paper extends the ideas of [5] by replacing info-fuzzy networks with neural networks as data mining and attribute selection tool.…”

Improving cascading classifiers with particle swarm optimization

“…In order to access the required computational effort of the cascading system we have used the total number of feature-values [6] which is given by…”

“…In order to speed up the optimization process, one particle of the swarm was initialized with the threshold produced by Chow's rule. Thereafter, the entire dynamics is governed by Equation 6.…”

Estimating the Serial Combination’s Performance from That of Individual Base Classifiers

“…Fusion of multiple classifiers can be performed in parallel or serially [1,2]. Although the most of works relies on parallel fusion at several levels (feature-level, measurement-level, decision-level) [3,4], very few papers deal with sequential, or serial, fusion of multiple classifiers [5][6][7][8][16][17][18].…”

“…Other works analyze specifically a specific Unfortunately, Pudil's modelling does not allow to point out any specific pros and cons of serial fusion of multiple classifiers. In other cases, we have practical applications of the serial fusion with an experimental assessment of pros and cons [6,7]. Theoretical and experimental approaches have been proposed for biometric applications [9][10][11].…”