A comparative analysis of feature selection stability measures

Chelvan, P Mohana; Perumal, K.

doi:10.1109/icoei.2017.8300901

Cited by 7 publications

(11 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Depending on the output of the feature selection algorithms, stability of feature selection measures are categorized into three groups. These are stability by rank, stability by weight and stability by similarity [12,13]. In the stability by rank approach, stability of feature selection algorithms, whose output is ranked lists of features, are evaluated by the correlation between two ranked feature lists.…”

Section: Stability Measures For Feature Selection Algorithmsmentioning

confidence: 99%

A Critical Study on Stability Measures of Feature Selection with a Novel Extension of Lustgarten Index

Sen

Mandal

Chakraborty

2021

MAKE

View full text Add to dashboard Cite

Stability of feature selection algorithm refers to its robustness to the perturbations of the training set, parameter settings or initialization. A stable feature selection algorithm is crucial for identifying the relevant feature subset of meaningful and interpretable features which is extremely important in the task of knowledge discovery. Though there are many stability measures reported in the literature for evaluating the stability of feature selection, none of them follows all the requisite properties of a stability measure. Among them, the Kuncheva index and its modifications, are widely used in practical problems. In this work, the merits and limitations of the Kuncheva index and its existing modifications (Lustgarten, Wald, nPOG/nPOGR, Nogueira ) are studied and analysed with respect to the requisite properties of stability measure. One more limitation of the most recent modified similarity measure, Nogueira’s measure, has been pointed out. Finally, corrections to Lustgarten’s measure have been proposed to define a new modified stability measure that satisfies the desired properties and overcomes the limitations of existing popular similarity based stability measures. The effectiveness of the newly modified Lustgarten’s measure has been evaluated with simple toy experiments.

show abstract

Section: Stability Measures For Feature Selection Algorithmsmentioning

confidence: 99%

A Critical Study on Stability Measures of Feature Selection with a Novel Extension of Lustgarten Index

Sen

Mandal

Chakraborty

2021

MAKE

View full text Add to dashboard Cite

show abstract

“…The second evaluation approach involves the stability of the ensemble feature selection itself. There are three categories for stability measurement, which are stability by index/subset, stability by rank, and stability by weight [38,39]. Stability by rank and weight has a major drawback that does not allow stability calculations on two subsets of features that have different numbers of features.…”

Section: Accuracy (Acc) =mentioning

confidence: 99%

“…rank, and stability by weight[38,39]. Stability by rank and weight has a major drawback that does 223 not allow stability calculations on two subsets of features that have different numbers of features.…”

mentioning

confidence: 99%

Optimal Feature Aggregation and Combination for Two-Dimensional Ensemble Feature Selection

Alhamidi

Jatmiko

2020

Information

View full text Add to dashboard Cite

Feature selection is a way of reducing the features of data such that, when the classification algorithm runs, it produces better accuracy. In general, conventional feature selection is quite unstable when faced with changing data characteristics. It would be inefficient to implement individual feature selection in some cases. Ensemble feature selection exists to overcome this problem. However, with the advantages of ensemble feature selection, some issues like stability, threshold, and feature aggregation still need to be overcome. We propose a new framework to deal with stability and feature aggregation. We also used an automatic threshold to see whether it was efficient or not; the results showed that the proposed method always produces the best performance in both accuracy and feature reduction. The accuracy comparison between the proposed method and other methods was 0.5–14% and reduced more features than other methods by 50%. The stability of the proposed method was also excellent, with an average of 0.9. However, when we applied the automatic threshold, there was no beneficial improvement compared to without an automatic threshold. Overall, the proposed method presented excellent performance compared to previous work and standard ReliefF.

show abstract

“…Next, Pearson's correlation coefficient is generally used to measure stability of feature weights in feature selection algorithms [9], but this is a measure of the general trend of importance and does not calculate the degree by which a feature's weight may vary. As such, we specify the measure stability by weight based on the statistical measure of relative variance, and calculate the stability of feature weights (φ(W)) for a single process instance in an event log as follows:…”

Section: Explanation Stabilitymentioning

confidence: 99%

Evaluating Explainable Methods for Predictive Process Analytics: A Functionally-Grounded Approach

Velmurugan¹,

Ouyang²,

Moreira³

et al. 2020

Preprint

View full text Add to dashboard Cite

Predictive process analytics focuses on predicting the future states of running instances of a business process. While advanced machine learning techniques have been used to increase accuracy of predictions, the resulting predictive models lack transparency. Current explainable machine learning methods, such as LIME and SHAP, can be used to interpret black box models. However, it is unclear how fit for purpose these methods are in explaining process predictive models. In this paper, we draw on evaluation measures used in the field of explainable AI and propose functionally-grounded evaluation metrics for assessing explainable methods in predictive process analytics. We apply the proposed metrics to evaluate the performance of LIME and SHAP in interpreting process predictive models built on XGBoost, which has been shown to be relatively accurate in process predictions. We conduct the evaluation using three open source, real-world event logs and analyse the evaluation results to derive insights. The research contributes to understanding the trustworthiness of explainable methods for predictive process analytics as a fundamental and key step towards human user-oriented evaluation.

show abstract

A comparative analysis of feature selection stability measures

Cited by 7 publications

References 5 publications

A Critical Study on Stability Measures of Feature Selection with a Novel Extension of Lustgarten Index

A Critical Study on Stability Measures of Feature Selection with a Novel Extension of Lustgarten Index

Optimal Feature Aggregation and Combination for Two-Dimensional Ensemble Feature Selection

Evaluating Explainable Methods for Predictive Process Analytics: A Functionally-Grounded Approach

Contact Info

Product

Resources

About