Some effects of fault recovery order on software reliability models

Abstract

Since traditional approaches to software reliability modeling allow the user to formulate predictions using data from one realization of the debugging process, it is necessary to understand the influence of the fault recovery order on predictive performance. We introduce an experimental methodology using a data structure called the debugging graph and use it to analyze the effects of various fault recovery orders on the predictive accuracy of four well-known sofrware reliability algorithms. Further we note fault interactions and their potential effects on the predictive process. Based on our experiment, we conjecture that the accuracy of a reliability prediction is affected by the fault recovery order as well as by fault interactions.Empirical Studies *This research was partially supported by NASA grant NAG-1-750.stage of the fault removal process. We investigate multiple orders for the fault removal process and isolate the effects of varying the recovery order by using an average of multiple observations to represent each interfailure time.It has been conjectured in the literature that one failure may prevent access to or hide certain others. The manifestation of this has been called the fault interaction phenomenon [3]. Other researchers claim that in practice, although such interactions may occasionally occur during unit testing, they are much less common during system testing or in the operational phase [ 6 ] . Our data display some manifestations of fault interactions.We believe that fault recovery order may affect the accuracy of predictions made by software reliability models, and that the effects of fault interactions are subsumed by the recovery order problem, since for a particular recovery order the context in which each fault contributes to the failure rate is fixed. Here we focus on the use of the debugging graph to analyze the effects of recovery order on reliability predictions. 1: Introduction 2: Terminology and modelsIt has been observed that predicting the reliability of a program has proved to be an unexpectedly challenging task for over two decades [l]. Also, life critical applications of software demand better performance and increased accuracy from reliability models. This has motivated us to investigate sources of inaccuracy in existing models with the hope of improving their performance and/or of discovering better models.Typically software reliability models use a sequence of interfailure times from the debugging process to predict reliability. However the sequence of interfailure times is derived from only one of many possible repair orders. If one assumes data from n failures are being used, then there are n! possible orders in which those faults could have been individually identified and repaired. Also uncertainty about the order of fault recovery is compounded in that a sample of size one is used to represent the interfailure time of the software for each Software reliability (R) is the probability of a software product operating for a given period of time in a particular environment...

An experiment in estimating reliability growth under both representative and directed testing

“…The final parameter values of the Jelinski-Moranda and Order Statistic models were largely unaffected by the debugging sequence, but the final parame- ter values for the Musa log model showed more variation, indicating that this model appears more sensitive to permutations in the debugging sequence. Such sensitivity has been reported by Hoppa and Wilson [6], though they found the Jelinski-Moranda model even more sensitive. Factors that may account for this difference are:…”

“…Although interactions among faults are known in practice, there is surprisingly little data indicating whether these interactions are statistically significant. Hoppa and Wilson have examined these interactions, but do not isolate their effects from differences in the fault detection order [6].…”

“…In practice, exact adherence to this order is unlikely. Hoppa and Wilson have argued that many models are extremely sensitive to permutations in the debugging sequence [6].…”

Software reliability accelerated testing method based on mixed testing

“…For the second question, the software reliability estimation is very sensitive to the sequence of the software failures [2], and Hoppa's study [2] also shows that the sequences of the software failures are quite different for the specified software according to several software reliability testing. So it is obvious that the sequences of the exposure of the software failures are different and it does affect the software reliability estimation if it is calculated by the classic software reliability models.…”

“…In most cases they do not follow the ideal sequence, but the number of the deviations is within limits [2], we still can make the equation (4) approximately true: …”