Toward integrating feature selection algorithms for classification and clustering

Abstract

Abstract-This paper introduces concepts and algorithms of feature selection, surveys existing feature selection algorithms for classification and clustering, groups and compares different algorithms with a categorizing framework based on search strategies, evaluation criteria, and data mining tasks, reveals unattempted combinations, and provides guidelines in selecting feature selection algorithms. With the categorizing framework, we continue our efforts toward building an integrated system for intelligent feature selection. A unifying platform is proposed as an intermediate step. An illustrative example is presented to show how existing feature selection algorithms can be integrated into a meta algorithm that can take advantage of individual algorithms. An added advantage of doing so is to help a user employ a suitable algorithm without knowing details of each algorithm. Some real-world applications are included to demonstrate the use of feature selection in data mining. We conclude this work by identifying trends and challenges of feature selection research and development.

Feature Selection for High Dimensional Data: An Evolutionary Filter Approach

“…1: subset generation, subset evaluation, stopping criterion and result validation (Liu and Yu, 2005). These steps are performed by three core components, namely, search algorithms, evaluation function and performance analyzer (Dash and Liu, 1997).…”

“…As a result, various feature selection approaches have accumulated over the years. To better underst and the inner instrument of each approach and the commonalities and differences between them, several taxonomies have emerged such as those proposed by Dash and Liu (1997), (Liu and Yu, 2005;Saeys et al, 2007). This study adopts the taxonomy depicted in Fig.…”

“…In spite of the vast body of feature selection approaches, the incessant increase of data dimensions (number of features) and data size (number of instances) poses sever challenges with respect to their efficiency and effectiveness (Liu and Yu, 2005;Zheng and Zhang, 2007). One of these challenges, which is that the focus of this study, is the so-called curse of dimensionality (Hastie et al, 2001).…”

“…The embedded approaches, on the other h ands, are specific to some ML algorithms, though, their time complexity is far less than wrapper approaches. It is, therefore, commonly accepted fact that the filter approaches are preferred for feature selection in high dimensional data due to their computational efficiency (Liu and Yu, 2005;Zheng and Zhang, 2007;Duch, 2006). Some examples of researches that use filter approach for feature selection in high dimensional domains are (Biesiada and Duch, 2005;Bins and Draper, 2001;Yu and Liu, 2003;Li et al, 2004;Guo et al, 2008).…”

“…Despite the striking success of GA, the increase of the data dimensionality poses a challenge to its straightforward application and consequently to its efficiency . Some attempts to address this challenge have been made by introducing a simplification assumption with respect to the number of features that must be selected (Liu and Yu, 2005;Hong and Cho, 2006;Sanchez-Ferrero and Arribas, 2007;Lecocke and Hess, 2007). Definitely, such assumption is not correct as the number of features that must be selected cannot be known a priori.…”