Performance Analysis of Feature Selection Methods in Software Defect Prediction: A Search Method Approach

Abstract

Software Defect Prediction (SDP) models are built using software metrics derived from software systems. The quality of SDP models depends largely on the quality of software metrics (dataset) used to build the SDP models. High dimensionality is one of the data quality problems that affect the performance of SDP models. Feature selection (FS) is a proven method for addressing the dimensionality problem. However, the choice of FS method for SDP is still a problem, as most of the empirical studies on FS methods for SDP produce contradictory and inconsistent quality outcomes. Those FS methods behave differently due to different underlining computational characteristics. This could be due to the choices of search methods used in FS because the impact of FS depends on the choice of search method. It is hence imperative to comparatively analyze the FS methods performance based on different search methods in SDP. In this paper, four filter feature ranking (FFR) and fourteen filter feature subset selection (FSS) methods were evaluated using four different classifiers over five software defect datasets obtained from the National Aeronautics and Space Administration (NASA) repository. The experimental analysis showed that the application of FS improves the predictive performance of classifiers and the performance of FS methods can vary across datasets and classifiers. In the FFR methods, Information Gain demonstrated the greatest improvements in the performance of the prediction models. In FSS methods, Consistency Feature Subset Selection based on Best First Search had the best influence on the prediction models. However, prediction models based on FFR proved to be more stable than those based on FSS methods. Hence, we conclude that FS methods improve the performance of SDP models, and that there is no single best FS method, as their performance varied according to datasets and the choice of the prediction model. However, we recommend the use of FFR methods as the prediction models based on FFR are more stable in terms of predictive performance.

A Classification Framework for Software Defect Prediction Using Multi-filter Feature Selection Technique and MLP

“…It has been reported by many researchers that in most of the datasets only few of the independent features can predict the target class effectively and remaining features don't only participate but can reduce the performance of classification model, if not removed. In this research, we have incorporated an aggregation based multi-filter feature selection technique, in which CFS [28,29,30] is used as attribute evaluator along with four widely used search methods including: GA, PSO, BFS, and FS. For each of the used dataset, feature selection is performed with all of these four search methods.…”

Software Defect Prediction Using Variant based Ensemble Learning and Feature Selection Techniques

“…As, it has been proved now from the studies [2], [5] that those features should be removed from the dataset which do not participate in classification process as these features may reduce the performance. In this research, feature selection is performed by "Cfs Subset Evaluator" [28,29,30] with BestFirst search method, whereas full dataset is given for training. In this approach, three directions are used for feature subset selection: Forward, Backward, and Bi-Directional.…”

Tree-based homogeneous ensemble model with feature selection for diabetic retinopathy prediction

“…Feature selection is a dimensionality reduction technique that involves selecting relevant and irredundant features from a dataset [24]. In this study, a filter-based feature selection method, Chi-Square (CS), was incorporated to reduce the number of features used in training the model.…”