Filter based backward elimination in wrapper based PSO for feature selection in classification

Abstract

The advances in data collection increase the dimensionality of the data (i.e. the total number of features) in many fields, which arises a challenge to many existing feature selection approaches. This paper develops a new feature selection approach based on particle swarm optimisation (PSO) and a local search that mimics the typical backward elimination feature selection method. The proposed algorithm uses a wrapper based fitness function, i.e. the classification error rate. The local search is performed only on the global best and uses a filter based measure, which aims to take the advantages of both filter and wrapper approaches. The proposed approach is tested and compared with three recent PSO based feature selection algorithms and two typical traditional feature selection methods. Experiments on eight benchmark datasets show that the proposed algorithm can be successfully used to select a significantly smaller number of features and simultaneously improve the classification performance over using all features. The proposed approach outperforms the three PSO based algorithms and the two traditional methods.

A Survey on Evolutionary Computation Approaches to Feature Selection

“…Multi-Objective Wrapper [10], [42], [70], [133], [134], [135], [136], [137], [138] [139], [140], [141], [142], [143], [144], [145], [146], [147], [148], [149], [150], [151], [152], [153], [154], [155], [156], [157], [158], [159], [160] [29], [161], [162] Filter [34], [163], [164], [165], [166], [167], [168], [169], [170] [171], [172], [173], [174] Combined [11], [33], [175], [176], [177] C. PSO for Feature Selection…”

“…GAs for wrapper feature selection and a local search using Markov blanket for filter feature selection. Similarly, local search for filter feature selection using mutual information was applied together with GAs and PSO for wrapper feature selection to develop memetic approaches in [50], [177] and [229]. A two-stage feature selection algorithm was proposed in [214], where a Relief-F algorithm was used to rank individual features and then the top-ranked features were used as input to the memetic wrapper feature selection algorithm.…”

“…(2) Use an information measure in the local search of a memetic algorithm. Mutual information [177], [227], symmetrical uncertainty [50], and Markov blanket [228] were used in local search to perform a filter feature selection to refine the solutions obtained by a GA or PSO for wrapper feature selection. (3) Incorporate an information measure into the updating/search mechanism.…”

Mutual Information Iterated Local Search: A Wrapper-Filter Hybrid for Feature Selection in Brain Computer Interfaces

“…Mutual information was used to reduce the search space in advance of running a Genetic Algorithm in Tan, Fu, Zhang, and Bourgeois [14] and Particle Swarm Optimisation in Ali and Shahzad [15]. It has also been successfully used within memetic algorithms as a local search method to refine the solutions found by PSO in Particle Swarm Optimisation Backwards Elimination (PSOBE) [16] and in Genetic Algorithms [17]. A common observation however, is that mutual information is almost always used as a local search operator in these cases, and to our knowledge, has not been used in the explorative phase of a metaheuristic prior to this paper.…”

Hybrid ACO-PSO Based Approaches for Feature Selection

“…Nguyen et al [51] propose a new feature selection approach based on particle swarm optimization (PSO). The local search is based on a typical backward elimination method to improve the gbest during the search process.…”