Ensemble feature selection for high-dimensional data: a stability analysis across multiple domains

Abstract

Selecting a subset of relevant features is crucial to the analysis of high-dimensional datasets coming from a number of application domains, such as biomedical data, document and image analysis. Since no single selection algorithm seems to be capable of ensuring optimal results in terms of both predictive performance and stability (i.e. robustness to changes in the input data), researchers have increasingly explored the effectiveness of ''ensemble'' approaches involving the combination of different selectors. While interesting proposals have been reported in the literature, most of them have been so far evaluated in a limited number of settings (e.g. with data from a single domain and in conjunction with specific selection approaches), leaving unanswered important questions about the large-scale applicability and utility of ensemble feature selection. To give a contribution to the field, this work presents an empirical study which encompasses different kinds of selection algorithms (filters and embedded methods, univariate and multivariate techniques) and different application domains. Specifically, we consider 18 classification tasks with heterogeneous characteristics (in terms of number of classes and instances-to-features ratio) and experimentally evaluate, for feature subsets of different cardinalities, the extent to which an ensemble approach turns out to be more robust than a single selector, thus providing useful insight for both researchers and practitioners.

Popularity Prediction of Instagram Posts

“…the weak learners, which are trained and combined together in order to solve specific problems. Indeed, this strategy relies on the idea that such a combination of several single (weak) models, in particular if associated with a proper feature selection step [16], can lead to an improvement of the final accuracy.…”

Measuring Success of Heterogeneous Ensemble Filter Feature Selection Models

“…Since then, many research work on ensemble feature selection is published. We can categorize most of the published work into four classes; (i) Work that focus on proofing that using ensemble filter feature selection models can improve the performance of machine learning rather than using single feature selection methods [ [18], and (iv) Research work that focuses on providing reviews and surveys for existing approaches and applications of ensemble filter feature selection models [19].…”

“…Experimental evaluations indicated that their proposed ensemble model is an efficient method, and it outperforms individual filter-based feature selection methods on sentiment classification. Similarly, an empirical exploration of the effectiveness of homogenous ensemble approach is studied [18]. To construct the model, a single feature selection algorithm is applied to several diversified datasets derived from the original set of records.…”

Naive Bayesian Machine Learning to Diagnose Breast Cancer

“…genes to reduce the number of features and dimensions. [10][11][12] A strength point of our work is that we consider ML as powerful advanced statistics tool doing heavy statistical analyses, that people themselves cannot do. As a result, we gave all the data corresponding to the WES as feature inputs to the ML at once and it returned almost perfect results quickly and precisely.…”