A Large-Scale Study of the Impact of Feature Selection Techniques on Defect Classification Models

Ghotra, Baljinder; McIntosh, Shane; Hassan, Ahmed E.

doi:10.1109/msr.2017.18

Cited by 97 publications

(117 citation statements)

References 48 publications

Supporting

Mentioning

103

Contrasting

Order By: Relevance

“…We also observe that wrapper-based feature selection techniques have the least impact on model performance for both classification techniques. This finding is consistent with Ghotra et al [24] who find that the performance of defect models is impacted by at most 2%pts for the AUC measure when applying wrapperbased feature selection techniques. We observe that filterbased feature selection techniques (except for consistency-based feature selection techniques) have the highest impact on the performance of defect models, since filter-based techniques tend to overly remove metrics that share a strong relationship with the outcome.…”

Section: An Evaluation Of Autospearmansupporting

confidence: 92%

“…Since it is impractical to study all of these techniques, we would like to select a manageable set of feature selection techniques for our study. Similar to Ghotra et al [24], we select two commonly-used families of feature selection techniques, i.e., filter-based feature selection techniques and wrapper-based feature selection techniques. Thus, embedded-based feature selection techniques are excluded from our analysis, as they are rarely explored in software engineering.…”

Section: Stepwise Regression (Both Directions)mentioning

confidence: 99%

“…Prior research effort has shown the benefits of applying feature selection techniques to defect prediction models [24,46,83]. For example, Ghotra et al [24], Lu et al [46], and Xu et al [83] investigate the impact of feature selection techniques on the performance of defect models. Thus, we set out to investigate the impact of AutoSpearman on the performance of defect models when comparing to the commonly-used feature selection techniques.…”

Section: An Evaluation Of Autospearmanmentioning

confidence: 99%

See 2 more Smart Citations

AutoSpearman: Automatically Mitigating Correlated Software Metrics for Interpreting Defect Models

Jiarpakdee

Tantithamthavorn

2018

2018 IEEE International Conference on Software Maintenance and Evolution (ICSME)

View full text Add to dashboard Cite

The interpretation of defect models heavily relies on software metrics that are used to construct them. However, such software metrics are often correlated in defect models. Prior work often uses feature selection techniques to remove correlated metrics in order to improve the performance of defect models. Yet, the interpretation of defect models may be misleading if feature selection techniques produce subsets of inconsistent and correlated metrics. In this paper, we investigate the consistency and correlation of the subsets of metrics that are produced by nine commonly-used feature selection techniques. Through a case study of 13 publicly-available defect datasets, we find that feature selection techniques produce inconsistent subsets of metrics and do not mitigate correlated metrics, suggesting that feature selection techniques should not be used and correlation analyses must be applied when the goal is model interpretation. Since correlation analyses often involve manual selection of metrics by a domain expert, we introduce AutoSpearman, an automated metric selection approach based on correlation analyses. Our evaluation indicates that AutoSpearman yields the highest consistency of subsets of metrics among training samples and mitigates correlated metrics, while impacting model performance by 1-2%pts. Thus, to automatically mitigate correlated metrics when interpreting defect models, we recommend future studies use AutoSpearman in lieu of commonly-used feature selection techniques.

show abstract

Section: An Evaluation Of Autospearmansupporting

confidence: 92%

Section: Stepwise Regression (Both Directions)mentioning

confidence: 99%

Section: An Evaluation Of Autospearmanmentioning

confidence: 99%

See 1 more Smart Citation

AutoSpearman: Automatically Mitigating Correlated Software Metrics for Interpreting Defect Models

Jiarpakdee

Tantithamthavorn

2018

2018 IEEE International Conference on Software Maintenance and Evolution (ICSME)

View full text Add to dashboard Cite

show abstract

“…The most discriminator terms could provide a deep insight into each feature. Recently, a large‐scale study conducted on 30 feature selection techniques and apply on 18 data sets. They found that correlation based ranking search technique perform well to select the important features.…”

Section: Methodsmentioning

confidence: 99%

Identifying and predicting key features to support bug reporting

Karim

Ihara

Choi

et al. 2019

J Software Evolu Process

View full text Add to dashboard Cite

Bug reports are the primary means through which developers triage and fix bugs. To achieve this effectively, bug reports need to clearly describe those features that are important for the developers. However, previous studies have found that reporters do not always provide such features. Therefore, we first perform an exploratory study to identify the key features that reporters frequently miss in their initial bug report submissions. Then, we propose an approach that predicts whether reporters should provide certain key features to ensure a good bug report. A case study of the bug reports for Camel, Derby, Wicket, Firefox, and Thunderbird projects shows that Steps to Reproduce, Test Case, Code Example, Stack Trace, and Expected Behavior are the additional features that reporters most often omit from their initial bug report submissions. We also find that these features significantly affect the bug‐fixing process. On the basis of our findings, we build and evaluate classification models using four different text‐classification techniques to predict key features by leveraging historical bug‐fixing knowledge. The evaluation results show that our models can effectively predict the key features. Our comparative study of different text‐classification techniques shows that naïve Bayes multinomial (NBM) outperforms other techniques. Our findings can benefit reporters to improve the contents of bug reports.

show abstract

“…We use principal component analysis (PCA) [8] to account for multi-collinearity amongst features [8], and has been extensively used in the domain of defect prediction [41] [42]. PCA creates independent linear combinations of the features that account for most of the co-variation of the features.…”

Section: Principal Component Analysismentioning

confidence: 99%

Source code properties of defective infrastructure as code scripts

Rahman

Williams

2019

Information and Software Technology

View full text Add to dashboard Cite

Context: In continuous deployment, software and services are rapidly deployed to end-users using an automated deployment pipeline. Defects in infrastructure as code (IaC) scripts can hinder the reliability of the automated deployment pipeline.We hypothesize that certain properties of IaC source code such as lines of code and hard-coded strings used as configuration values, show correlation with defective IaC scripts.Objective: The objective of this paper is to help practitioners in increasing the quality of infrastructure as code (IaC) scripts through an empirical study that identifies source code properties of defective IaC scripts.Methodology: We apply qualitative analysis on defect-related commits mined from open source software repositories to identify source code properties that correlate with defective IaC scripts. Next, we survey practitioners to assess the practitioner's agreement level with the identified properties. We also construct defect prediction models using the identified properties for 2,439 scripts collected from four datasets.Results: We identify 10 source code properties that correlate with defective IaC scripts. Of the identified 10 properties we observe lines of code and hard-coded string to show the strongest correlation with defective IaC scripts. Hard-coded string is the property of specifying configuration value as hard-coded string. According to our survey analysis, majority of the practitioners show agreement for two properties: include, the property of executing external modules or scripts, and hard-coded string. Using the identified properties, our constructed defect prediction models show a precision of 0.70∼0.78, and a recall of 0.54∼0.67.Conclusion: Based on our findings, we recommend practitioners to allocate sufficient inspection and testing efforts on IaC scripts that include any of the identified 10 source code properties of IaC scripts.

show abstract

A Large-Scale Study of the Impact of Feature Selection Techniques on Defect Classification Models

Cited by 97 publications

References 48 publications

AutoSpearman: Automatically Mitigating Correlated Software Metrics for Interpreting Defect Models

AutoSpearman: Automatically Mitigating Correlated Software Metrics for Interpreting Defect Models

Identifying and predicting key features to support bug reporting

Source code properties of defective infrastructure as code scripts

Contact Info

Product

Resources

About