Software reliability growth with test coverage

Abstract

Software test-coverage measures" quantify the degree of thoroughness of testing. Tools are now available that measure test-coverage in terms of blocks, branches, computation-uses, predicate-uses, etc. that are covered. This paper models the relations among testing time, coverage, and reliability. An LE (logarithmic-exponential) model is presented that relates testing effort to test coverage (block, branch, computation-use, or predicate-use). The model is based on the hypothesis that the enumerable elements (like branches or blocks) for any coverage measure have various probabilities of being exercised; just like defects have various probabilities of being encountered. This model allows relating a test-coverage measure directly with defect-coverage. The model is fitted to 4 data-sets for programs with real defects. In the model, defect coverage can predict the time to next failure. The LE model can eliminate variables like test-application strategy from consideration. It is suitable for high reliability applications where automatic (or manual) test generation is used to cover enumerables which have not yet been tested. The data-sets used suggest the potential of the proposed model. The model is simple and easily explained, and thus can be suitable for industrial use. The LE model is based on the time-based logarithmic software-reliability growth model. It considers that: at 100% coverage for a given enumerable, all defects might not yet have been found.

Architecture-Based Software Reliability Analysis: Overview and Limitations

“…Code coverage measurements may also be used to estimate the failure parameters of the components [7], [43], [62], [26]. To estimate fault propagation probabilities among the components, a fault injection technique may be used [91], [46].…”

Virtual Coverage: A New Approach to Coverage-Based Software Reliability Engineering

“…Numerous researchers have attempted to integrate coverage information into SRGMs. Gokhale et al (1996), Malaiya et al (2002), Pham and Zhang (2003), Kapur et al (2006) and Park et al (2008b) studied SRGMs taking a coverage growth function(CGF) into account. These SRGMs are called the coverage-based SRGMs.…”

Estimation of Coverage Growth Functions

“…Such a relationship is obtained by modeling the fault detection and removal process during testing; however, it was recognized that the testing time was not enough to express the fault detection and removal process. Attempts to integrate coverage information into SRGMs have been made by Gokhale et al (1996), Malaiya et al (2002), Pham and Zhang (2003), Park and Fujiwara (2006), Crespo et al (2008, 2009), and Park et al (2008b. Each coverage-based SRGM involves a coverage growth function(CGF), that describes the coverage growth process during testing.…”