A controlled experiment in program testing and code walkthroughs/inspections

Abstract

This paper describes an experiment in program testing, employing 59 highly experienced data processing professionals using seven methods to test a small PL/I program. The results show that the popular code walkthrough/inspection method was as effective as other computer-based methods in finding errors and that the most effective methods (in terms of errors found and cost) employed pairs of subjects who tested the program independently and then pooled their findings. The study also shows that there is a tremendous amount of variability among subjects and that the ability to detect certain types of errors varies from method to method.

Functional Testing, Structural Testing and Code Reading: What Fault Type Do They Each Detect?

“…The roots of this study go back to the work of Hetzel [3] and Myers [7]. More precisely, it is the continuation of a line of experiments run by other authors, which have added to the knowledge provided by previous experiments.…”

Evolving and Packaging Reading Technologies

“…To some extent, this might explain the success of reading in this experiment over the studies by Hetzel (Hetzel 1972) and Myers (Myers 1978).…”

“…One area of interest was to understand the relationship between reading and testing in our environment. Early experiments showed very little difference between reading and testing (Hetzel1972; Myers 1978). But reading in these studies was simply reading, without a technological base.…”

Improving the Software Inspection Process

“…Already in 1978, Meyers run an experiment at the IBM Systems Research Institute where he compared testing and inspections [4]. The experiment showed no significant differences between the methods when it comes to error detection efficiency but testing showed a higher variability than inspection.…”