Design for instrumentation: high quality measurement of formal technical review

Abstract

All current software quality assurance methods incorporate some form of formal technical review (FTR), because structured analysis of software artifacts by a team of skilled technical personnel has demonstrated ability to improve quality. However, FTR methods come in a wide variety of forms with varying effectiveness, incur significant overhead on technical staff, and have little computer support. Measurements of these FTR methods are coarse-grained, frequently low quality, and expensive to obtain. This paper describes CSRS, a highly instrumented, computer-supported system for formal technical review, and shows how it is designed to collect high quality, fine-grained measures of FTR process and products automatically. The paper also discusses some results from over one year of experimentation with CSRS; describes how CSRS improves current process improvement approaches to FTR; and overviews several novel research projects on FTR that are made possible by this system.