Statistical Analysis of the Performance of Rank Fusion Methods Applied to a Homogeneous Ensemble Feature Ranking

Abstract

The feature ranking as a subcategory of the feature selection is an essential preprocessing technique that ranks all features of a dataset such that many important features denote a lot of information. The ensemble learning has two advantages. First, it has been based on the assumption that combining different model’s output can lead to a better outcome than the output of any individual models. Second, scalability is an intrinsic characteristic that is so crucial in coping with a large scale dataset. In this paper, a homogeneous ensemble feature ranking algorithm is considered, and the nine rank fusion methods used in this algorithm are analyzed comparatively. The experimental studies are performed on real six medium datasets, and the area under the feature-forward-addition curve criterion is assessed. Finally, the statistical analysis by repeated-measures analysis of variance results reveals that there is no big difference in the performance of the rank fusion methods applied in a homogeneous ensemble feature ranking; however, this difference is a statistical significance, and the B-Min method has a little better performance.

Distributed Ensemble Feature Selection Framework for High-Dimensional and High-Skewed Imbalanced Big Dataset

“…The DEIM framework can utilize an arbitrary feature ranking algorithm, such as Fisher, and Gini Index [21], as the base learner in phase three. Nevertheless, algorithms that belong to the filter category would be better than others because they have lower computational costs and more generality [22].…”

“…This capability leads to lower execution time than the traditional version, whereas the outcome was not destroyed. Moreover, the author's other work investigated the various rank combination methods in an ensemble feature selection approach [21]. Their proposed method focused on scalability and combining feature rankings, whereas coping with imbalanced datasets was not investigated.…”

“…At the end of phase three, each data partition is transformed to the b number of intermediate feature rankings, which should be reduced to a final feature ranking by applying a rank fusion method. The rank fusion is known by various names, such as Rank Combination and Rank Aggregation as well, and some methods such as Borda, Kwik-Sort, and Stuart are introduced in this field [21]. Due to some advantages, the proposed framework applies an Ordered Weighted Averaging (OWA) operator to reduce the intermediate feature rankings.…”

Scalable Global Mutual Information Based Feature Selection Framework for Large Scale Datasets

“…The noticeable point is that computing the individual entropy and joint entropy would be feasible by generating the joint value histogram vector. Therefore, if the joint value histogram of two features X i and X j is represented as a square matrix F ∈ N b×b such that f i,j is equal to the frequency of joint values (x i , x j ), the joint and individual entropies will be computed based on equations (11) to (13). Therefore, acquiring the joint value histogram of two features, the MI measure between them can be computed in a single pass.…”

“…According to dependency on a classification model, FS algorithms can be categorized into three groups: wrapper methods [8], [9], embedded methods [10], and filter methods [11]. Filter methods only rest on data's statistical properties, and since they are independent of any learning model, they prevent incurring a high computational cost and provide more generality than two other categories [12].…”