AutoSpearman: Automatically Mitigating Correlated Software Metrics for Interpreting Defect Models

Abstract

The interpretation of defect models heavily relies on software metrics that are used to construct them. However, such software metrics are often correlated in defect models. Prior work often uses feature selection techniques to remove correlated metrics in order to improve the performance of defect models. Yet, the interpretation of defect models may be misleading if feature selection techniques produce subsets of inconsistent and correlated metrics. In this paper, we investigate the consistency and correlation of the subsets of metrics that are produced by nine commonly-used feature selection techniques. Through a case study of 13 publicly-available defect datasets, we find that feature selection techniques produce inconsistent subsets of metrics and do not mitigate correlated metrics, suggesting that feature selection techniques should not be used and correlation analyses must be applied when the goal is model interpretation. Since correlation analyses often involve manual selection of metrics by a domain expert, we introduce AutoSpearman, an automated metric selection approach based on correlation analyses. Our evaluation indicates that AutoSpearman yields the highest consistency of subsets of metrics among training samples and mitigates correlated metrics, while impacting model performance by 1-2%pts. Thus, to automatically mitigate correlated metrics when interpreting defect models, we recommend future studies use AutoSpearman in lieu of commonly-used feature selection techniques.

Contextual Documentation Referencing on Stack Overflow

“…As Table 10 shows, some of our features are likely to be correlated, e.g., LineCountText and LengthText. To mitigate correlated metrics, we used AutoSpearman [35], an automated metric selection approach based on correlation analyses, with a threshold of 0.7.…”

The Impact of Correlated Metrics on the Interpretation of Defect Models

“…Prior work points out that software metrics are often correlated [22,32,33,35,36,74,77,85]. However, little is known the prevalence of correlated metrics in the publiclyavailable defect datasets.…”

“…Metrics of prior studies are often correlated [22,32,33,35,36,74,77,85]. For example, Herraiz et al [33], and Gil et al [22] point out that code complexity (CC) is often correlated with code size (size).…”

A Framework for Feature Selection to Exploit Feature Group Structures

“…Among the three main feature selection methods, filter methods are preferred to wrapper and embedded methods in applications where the computational efficiency, classifier independence, simplicity, ease of use and the stability of the results are required. Therefore, filter feature selection remains an interesting topic in many recent research areas such as biomarker identification for cancer prediction and drugs discovery, text classification and predicting defective software [3][4][5]10,11,16,18] and has growing interest in big data applications [19]; according to the Google Scholar search results, the number of research papers published related to filter methods in year 2018 is ∼1,800 of which ∼170 are in gene selection area.…”

“…The nearby pixels in images can be grouped together based on their spatial locality to improve selection of pixels for image classification. In software data, software metrics can be grouped according to their granularity in the code to improve the prediction of defective software [11,18]. In Sect.…”