Feature and instance selection through discriminant analysis criteria

Abstract

Feature selection and instance selection are two data preprocessing methods widely used in data mining and pattern recognition. The main goal is to reduce the computational cost of many learning tasks. Recently, joint feature and instance selection has been approached by solving some global optimization problems using meta-heuristics. This approach is not only computationally expensive, but also does not exploit the fact that the data usually has a structured manifold implicitly hidden in the data and its labels.In this paper, we address joint feature and instance selection using scores derived from discriminant analysis theory. We present three approaches for joint feature and instance selection.The first scheme is a wrapper technique, while the other two schemes are filtering techniques.In the filtering approaches, the search process uses a genetic algorithm where the evaluation criterion is mainly given by the discriminant analysis score. This score depends simultaneously on the feature subset candidate and the best corresponding subset of instances. Thus, the best feature subset and the best instances are determined by finding the best score. The performance of the proposed approaches is quantified and studied using image classification with Nearest Neighbor and Support Vector Machine Classifiers. Experiments are conducted on five public image datasets. We compare the performance of our proposed methods with several state-ofthe-art methods. The experiments performed show the superiority of the proposed methods over several baseline methods.

Feature importance feedback with Deep Q process in ensemble-based metaheuristic feature selection algorithms

“…Filter methods 1 , 2 are the most straightforward, ranking features based on their statistical properties. Metrics like chi-squared test, correlation coefficient, and mutual information are commonly used for different use cases 3 , 4 . However, these methods are univariate, examining each feature in isolation, which means they might overlook features that are useful in combinations.…”