Transparent combination of expert and measurement data for defect prediction

Abstract

Defining strategies on how to perform quality assurance (QA) and how to control such activities is a challenging task for organizations developing or maintaining software and softwareintensive systems. Planning and adjusting QA activities could benefit from accurate estimations of the expected defect content of relevant artifacts and the effectiveness of important quality assurance activities. Combining expert opinion with commonly available measurement data in a hybrid way promises to overcome the weaknesses of purely data-driven or purely expertbased estimation methods. This article presents a case study of the hybrid estimation method HyDEEP for estimating defect content and QA effectiveness in the telecommunication domain. The specific focus of this case study is the use of the method for gaining quantitative predictions. This aspect has not been empirically analyzed in previous work. Among other things, the results show that for defect content estimation, the method performs significantly better statistically than purely data-based methods, with a relative error of 0.3 on average (MMRE).

Transfer defect learning

“…One alternative would be 10-fold cross validation, but it essentially uses 90% training data and 10% test data. Moreover, random split is widely used in the literature [5], [9], [10].…”

“…Most approaches employ machine learning classifiers to build a prediction model from data sets mined from software repositories, and the model is used to identify software defects. However, most approaches are evaluated in within-project settings, i.e., a prediction model is built from a part of a project and the model is evaluated with the remainder of the project by 10-fold cross validation [2], [6], [7], [8] and/or random instance splits [5], [9], [10].…”

Heterogeneous defect prediction

“…We use 50:50 random splits, which are widely used in the evaluation of defect prediction models [22,37,41]. For the 50:50 random splits, we use one half of the instances for training a model and the rest for test (round 1).…”

Pre-Silicon Bug Forecast

“…In our study, code characteristics are chosen independent of specific hardware description languages, which is listed in Table I. 2) History Characteristics: History information, such as past changes, fixes, bugs, and so on, may also have significant impacts on bug occurrence. In the domain of software engineering, history information has already been demonstrated to be helpful in predicting software defects [20]- [22], [26], [31], [42]. Hence, the proposed pre-silicon bug forecast framework also utilizes history information.…”

“…In the field of software engineering, many studies have been dedicated to characterize the relationship between the software characteristics and fault-proneness to assess the design quality [5], [19]- [22], [26], [27], [30], [31], [42], which mainly focused on selecting characteristics that have most impacts on the fault-proneness of software.…”