Induction of strong feature subsets

Abstract

A b s t r a c t The problem of features subset selection can be defined as the selection of a relevant subset of features which allows a leanaing algorithm to induce small high-accuracy concepts. To achieve this goal, two main approaches have been developed, the first one is algorithm-independent (filter approach) which considers only the data, when the second approach takes into account both the data and a given learning algorithm (wrapper approach). Recent work were developed to study the interest of rough set theory and more particularly its notions of reducts and core to deal with the problem of features subset selection. Different methods were proposed to select features using both the core and the reduct concepts, whereas other researches show that useful features subset does not necessarily contain all features in cores. In this paper, we underline the fact that rough set theory is concerned with deterministic analysis of attribute dependencies which are at basis of the two notions of reduct and core. We extend the notion of dependency which allows us to find both deterministic and non-deterministic dependencies. A new notion of strong reducts is then introduced and leads to the definition of strong feature subsets (SFS). The interest of SFS is illustrated by the improvement of the accuracy of our learning system, called Alpha, on real-world datasets.

A hybrid approach to feature selection

“…: when alpha increases, the size of reducts decreases and the number of reducts generally increases; (2) we obtain the classical reducts (corresponding to classical framework of rough sets) when a=0; (3) the best reduct with highest accuracy is obtained when a > 0 i.e. using a-RST concepts [16]; (4) theoretically, the algorithm can find N ([N/2J) reducts.…”

“…In general, they can be classified into two categories: (1) the filter approach, which serves as a filter to sieve the irrelevant and/or redundant features without taking into account the induction algorithm [1][5] [10]; and (2) the wrapper approach, which uses the induction algorithm itself as a black box in the phase of attributes selection to select a good features subset which improve the performance, i.e. the accuracy of the induction algorithm [4] [8][11] [16]. Although the wrapper approach has significantly improved the accuracy of well known algorithms, like C4.5 and Naive-Bayes, its generalization is limited for many reasons i.e.…”

“…Kohavi and Frasca in [7] have shown that, in some situations, the useful subset does not necessarily contain all the features in the core and may be different from a reduct. Using a-RST, we have proposed in [16] an algorithm based on wrapper approach to solve this problem. We have shown that we can obtain lower size reducts with higher accuracy than those obtained by classic rough sets concepts.…”