A Heuristic Rule Reduction Approach to Software Fault-proneness Prediction

Abstract

Abstract-Background: Association rules are more comprehensive and understandable than fault-prone module predictors (such as logistic regression model, random forest and support vector machine). One of the challenges is that there are usually too many similar rules to be extracted by the rule mining. Aim: This paper proposes a rule reduction technique that can eliminate complex (long) and/or similar rules without sacrificing the prediction performance as much as possible. Method: The notion of the method is to removing long and similar rules unless their confidence level as a heuristic is high enough than shorter rules. For example, it starts with selecting rules with shortest length (length=1), and then it continues through the 2nd shortest rules selection (length=2) based on the current confidence level, this process is repeated on the selection for longer rules until no rules are worth included. Result: An empirical experiment has been conducted with the Mylyn and Eclipse PDE datasets. The result of the Mylyn dataset showed the proposed method was able to reduce the number of rules from 1347 down to 13, while the delta of the prediction performance was only .015 (from .757 down to .742) in terms of the F1 prediction criteria. In the experiment with Eclipsed PDE dataset, the proposed method reduced the number of rules from 398 to 12, while the prediction performance even improved (from .426 to .441.) Conclusion: The novel technique introduced resolves the rule explosion problem in association rule mining for software proneness prediction, which is significant and provides better understanding of the causes of faulty modules.

Cross-Validation-Based Association Rule Prioritization Metric for Software Defect Characterization

“…In a future work, we are planning to conduct experiments with a broader range of data sets to evaluate the generality of our approach. In addition, we are planning to combine our rule prioritization approach with rule reduction [1] to better identify useful association rules. number 17K00102.…”

“…Several various defect prediction models have already been proposed [7]- [11]. However, our focus in this paper is on understanding rather than predicting defects because previous predictive models have been difficult for humans to understand, making it hard for software engineers to recognize and agree why certain modules are (or are not) faulty [1]. Even with simple linear discriminant models, correlations between predictor variables make it difficult to interpret their coefficients clearly.…”

“…it may satisfy the antecedents of multiple rules. To handle this, we take the approach described in [1], which keeps the rule set small and understandable by only mining rules whose consequent is "faulty" and ignoring the rest. If at least one of these rules matches a module, then we consider it to be faulty; otherwise, we consider it not to be faulty; otherwise, we consider it not to be faulty.…”

“…Association rule mining discovers relationships among variables in a given data set and represents them as association rules. In the software engineering field, it has been applied to discover rules in software defect data sets [1]- [4], such as "(max nest level > 5) and (fan-out > 5) ⇒ faulty". This rule implies that a software module is likely to contain a defect (i.e.…”

“…In contrast, rules that have little predictive power are fairly useless, and using them to plan actions may even be harmful. In our case, we want to select the most predictive rules to better understand the characteristics of defect-prone modules in previous software product versions so that modules in the next version with similar characteristics can be thoroughly tested or inspected before release [1]. This paper proposes a cross-validation-based metric to quantify these rules' predictive power and hence prioritize them.…”