An approach to measuring and assessing dependability for critical software systems

Abstract

Traditional software testing methods combined krsith probabilistic models cannot nieasure arid assess dependability for software that requires v e n high reliabilio (failure rate < 10-'/hour) and availabilit?. (>0. 999999). This paper proposes a novel approach, drawing on findings and methods that have been described individriull~ hut have never been combined, applied in the late testing phase or early operational phase, to quantifi depetitlubility for U categon of critical sofhvare with such high requirenietits. The concepts that are integrated ure: operational profle, rare conditions, importance sampling, stress testing, and measurement-based dependabilir). evaluation. In the approach, importance sanipling is applied on the operational profile to guide the testing of critical operations of the software, thereby accelerating the occurrence of rare conditions which have been shown to be a leading cause of failure in critical systems. The failure rates measured in the testing are then transformed to those that would occur in the normal operation by the likelihood ratio function of the importance sanipling theory, and finally depetidabilih for the tested software system is evaluated by using measurement-based dependabilio modeling techniques. When the acceleration factor is large (over loo), which is r)pical for a category of software of interest, it is possible to quantib a very high reliability or availability in a reasonable test duration. Some feasible methods to inipletnent the approach are discussed based on real data.

Assessing the reliability of the hardware and software complex of fault-tolerant control systems

“…Therefore, software-induced failure also needs to be considered and included in the control system reliability model. To this end, we expand equation (3) to take into account possible software failures: 𝜆 ∑ 𝜆 т 𝜆 (4) where 𝜆 -is the software failure rate, which is determined by experimental software testing, based on the operational profile of the control system software [12].…”

An empirical study of injected versus actual interface errors

“…Kao et al [23] performed a Markov reward analysis, based on fault injection experiments, to quantify the expected impact of faults on performance and availability. Tang and Hetch [39] proposed an approach for accelerating the probabilistic evaluation of high-reliability systems (e.g., with a failure rate in the order of 10 −6 ) that adopts fault injection to force the occurrence of rare events. In [42], Voas et al inject errors within a program to identify where to place assertions and to avoid error propagation.…”

Software reliability accelerated testing based on the combined testing method

“…There have two methods which can be used to estimate the software reliability. Firstly, only based on the data collected from the second stage testing, the software reliability in the accelerated operation can be estimated, and then we can obtain the software reliability in the normal operation with the accelerated factor [9]. Secondly, we use the accelerated factor to become the data collected from the second stage testing to the failure data in the normal operation, and then to estimate the software reliability uses all the failure data.…”