An internally replicated quasi-experimental comparison of checklist and perspective based reading of code documents

Abstract

ÐThe basic premise of software inspections is that they detect and remove defects before they propagate to subsequent development phases where their detection and correction cost escalates. To exploit their full potential, software inspections must call for a close and strict examination of the inspected artifact. For this, reading techniques for defect detection may be helpful since these techniques tell inspection participants what to look for and, more importantly, how to scrutinize a software artifact in a systematic manner. Recent research efforts investigated the benefits of scenario-based reading techniques. A major finding has been that these techniques help inspection teams find more defects than existing state-of-the-practice approaches, such as, ad-hoc or checklist-based reading (CBR). In this paper, we experimentally compare one scenario-based reading technique, namely, perspective-based reading (PBR), for defect detection in code documents with the more traditional CBR approach. The comparison was performed in a series of three studies, as a quasi experiment and two internal replications, with a total of 60 professional software developers at Bosch Telecom GmbH. Meta-analytic techniques were applied to analyze the data. Our results indicate that PBR is more effective than CBR (i.e., it resulted in inspection teams detecting more unique defects than CBR) and that the cost of defect detection using PBR is significantly lower than CBR. Therefore, this study provides evidence demonstrating the efficacy of PBR scenarios for code documents in an industrial setting. Index TermsÐSoftware inspection, perspective-based reading, quasi experiment, replication, meta-analysis. ae 1. This constitutes the costs associated with correcting defects. 2. Both of these figures assume that the defect detection life cycle prior to the introduction of inspections consisted only of testing activities. 3. In this article, we model the inspection process in terms of its main activities. This allows us to be independent from a specific inspection implementation, such as the Fagan [24] or the Gilb [30] one.

Defect Types and Software Inspection Techniques: A Systematic Mapping Study

“…Thus, such common defect types existing in the literature occurs in the studies of Brykczynski (1999;Biffl and Halling, 2000;Laitenberger et al, 2001;Sabaliauskaite et al, 2002a;2004;Thelin et al, 2003;Denger and Paech, 2004;Cox et al, 2004a;Staron et al, 2005;Mohammed et al, 2015). A brief discussion of each study of such techniques/approaches is presented as follows:…”

“…In summary, the studies presented in this section answered RQ2 and provide evidence of the existing inspection techniques or approaches associated with defect types, encompassing studies, such as: (i) several academic validation studies that compare and evaluate the effectiveness of the main inspection techniques (e.g., CBR, PBR, SBR, Ad hoc, among others) in distinct contexts (Biffl and Halling, 2000;Cox et al 2004a;Denger and Paech, 2004;Laitenberger et al, 2001;Mohammed et al, 2015;Sabaliauskaite et al, 2002a;2003;Staron et al, 2005;Thelin et al, 2003); (ii) case studies in industry (Felderer et al, 2014;Ma et al, 2014; and (iii) application and adaptation of systematic classifications such as ODC (Gopinath et al, 2014;Ma et al 2014;.…”

“…The quasi-experiments performed by Laitenberger et al (2001) also established comparisons to identify the effectiveness and cost of defects detection in the PBR technique with respect to CBR. In order to evaluate this scenario, two replications were carried out taking practitioners from Bosch Telecom GmbH into consideration.…”

Assessing the impact of active guidance for defect detection: a replicated experiment

“…However, the results of these experiments do not give a conclusive answer to the research question. Some experiments showed that SBR techniques are more effective and efficient than CBR [3,15,18,22,23,26], while other experiments failed to show any significant difference between the techniques [9,11,20,24].…”

A Framework for Software Engineering Experimental Replications

“…A large body of studies has demonstrated that reading techniques are effective for improving individual review practices in different domains and types of inspection, e.g. natural language requirements [3], requirements in formal notation [20], high-level designs [14], code [13,15], and user interfaces [32].…”