Improving the Efficiency of Software Reliability Demonstration Testing by Introducing Testing Effectiveness

Abstract

For highly reliable software systems, it is expensive, time consuming, or even infeasible to perform reliability testing via a conventional software reliability demonstration testing (SRDT) plan. Moreover, in the traditional SRDT approach, the various characteristics of the software system or test sets are not considered when making the testing schemes. Some studies have focused on the effect of software testability on SRDT, but only limited situations were discussed, and many theoretical and practical problems have been left unresolved. In this paper, an extended study on the quantitative relation between test effectiveness (TE) and test effort for SRDT is proposed. Theoretical derivation is put forward by performing statistical analysis for the test suite according to TE. The combinations of all the cases of zero-failure and revealed nonzero failure, as well as discrete-type software and continuous-type software, are studied with the corresponding failure probability models constructed. That is, zero-failure and nonzero failure, as well as discrete-type software and continuous-type software, respectively, constitute the symmetry and asymmetry of SRDT. Finally, we illustrated all the models and performed applications on the Siemens program suite. The experimental results show that within the same limitation of requirements and confidence levels, this approach can effectively reduce the number of test cases and the test duration, i.e., accelerate the test process and improve the efficiency of the SRDT.