Software Fault-Freeness and Reliability Predictions

Strigini, Lorenzo; Povyakalo, Andrey

doi:10.1007/978-3-642-40793-2_10

Cited by 27 publications

(61 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The essential idea of CBI is applicable in a variety of contexts and scenarios [22][23][24][25][26]. It has been investigated for various objective functions (the posterior measures of interest) with different forms of constraints (the partial prior knowledge), e.g., a posterior expected failure rate given a prior confidence bound in [22].…”

Section: The Cbi As a Constant Event-rate Modelmentioning

confidence: 99%

“…For AVs, occasional failures are to be expected. Including operational evidence with "rare failures" into the assessment generalises existing CBI methods (applied in other settings such as nuclear safety) that, so far, consider only failurefree evidence [22][23][24][25][26]. Being a Bayesian approach, CBI allows for the incorporation of prior knowledge of non-road-testing evidence (e.g., verified aspects of the behaviour of an AV's ML algorithms; verification results for the safety subsystems).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Assessing safety-critical systems from operational testing: A study on autonomous vehicles

Zhao

Salako

Strigini

et al. 2020

Information and Software Technology

Self Cite

View full text Add to dashboard Cite

Context: Demonstrating high reliability and safety for safety-critical systems (SCSs) remains a hard problem. Diverse evidence needs to be combined in a rigorous way: in particular, results of operational testing with other evidence from design and verification. Growing use of machine learning in SCSs, by precluding most established methods for gaining assurance, makes evidence from operational testing even more important for supporting safety and reliability claims. Objective: We revisit the problem of using operational testing to demonstrate high reliability. We use Autonomous Vehicles (AVs) as a current example. AVs are making their debut on public roads: methods for assessing whether an AV is safe enough are urgently needed. We demonstrate how to answer 5 questions that would arise in assessing an AV type, starting with those proposed by a highly-cited study. Method: We apply new theorems extending our Conservative Bayesian Inference (CBI) approach, which exploit the rigour of Bayesian methods while reducing the risk of involuntary misuse associated (we argue) with now-common applications of Bayesian inference; we define additional conditions needed for applying these methods to AVs. Results: Prior knowledge can bring substantial advantages if the AV design allows strong expectations of safety before road testing. We also show how naive attempts at conservative assessment may lead to over-optimism instead; why extrapolating the trend of disengagements (take-overs by human drivers) is not suitable for safety claims; use of knowledge that an AV has moved to a "less stressful" environment. Conclusion: While some reliability targets will remain too high to be practically verifiable, our CBI approach removes a major source of doubt: it allows use of prior knowledge without inducing dangerously optimistic biases. For certain ranges of required reliability and prior beliefs, CBI thus supports feasible, sound arguments. Useful conservative claims can be derived from limited prior knowledge.

show abstract

Section: The Cbi As a Constant Event-rate Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Assessing safety-critical systems from operational testing: A study on autonomous vehicles

Zhao

Salako

Strigini

et al. 2020

Information and Software Technology

Self Cite

View full text Add to dashboard Cite

show abstract

“…We could however conclude that their pfd is so close to 0 that for predicting reliability over some number N'<N of future demands, they can be assumed 0 without substantial error. For a more complete study of "effective fault-freeness" see [21]. 9 These numbers are obtained by solving the familiar formulas for the probability of failure of redundant systems with independent failures, applied to independent processes of version development rather than to the versions themselves.…”

Section: E Measures Of Risk: Risk Of Exceeding a Required Pfdmentioning

confidence: 99%

Software Diversity as a Measure for Reducing Development Risk

Popov

Povyakalo

Stanković

et al. 2014

2014 Tenth European Dependable Computing Conference

Self Cite

View full text Add to dashboard Cite

“…• if instead we assume that a pfd equal to 0 is impossible, rather than difficult to achieve and demonstrate, then to demonstrate low probability of dangerous failure over a long operational life 12 we would need to test for some (often impractical) multiple of that life, or to have implausibly strong prior beliefs [33,34,41].…”

Section: Probability Of Pfd =mentioning

confidence: 99%

Bounds on survival probability given mean probability of failure per demand; and the paradoxical advantages of uncertainty

Strigini

Wright

2014

Reliability Engineering & System Safety

Self Cite

View full text Add to dashboard Cite

When deciding whether to accept into service a new safety-critical system, or choosing between alternative systems, uncertainty about the parameters that affect future failure probability may be a major problem. This uncertainty can be extreme if there is the possibility of unknown design errors (e.g. in software), or wide variation between nominally equivalent components.We study the effect of parameter uncertainty on future reliability (survival probability), for systems required to have low risk of even only one failure or accident over the long term (e.g. their whole operational lifetime) and characterised by a single reliability parameter (e.g. probability of failure per demand -pfd ). A complete mathematical treatment requires stating a probability distribution for any parameter with uncertain value. This is hard, so calculations are often performed using point estimates, like the expected value.We investigate conditions under which such simplified descriptions yield reliability values that are sure to be pessimistic (or optimistic) bounds for a prediction based on the true distribution. Two important observations are: (i) using the expected value of the reliability parameter as its true value guarantees a pessimistic estimate of reliability, a useful property in most safety-related decisions; (ii) with a given expected pfd, broader distributions (in a formally defined meaning of "broader"), that is, systems that are a priori "less predictable", lower the risk of failures or accidents.Result (i) justifies the simplification of using a mean in reliability modelling; we discuss within which scope this justification applies, and explore related scenarios, e.g. how things improve if we can test the system before operation. Result (ii) offers more flexible ways of bounding reliability predictions, but also has important, often counter-intuitive implications for decision making in various areas, like selection of components, project management, and product acceptance or licensing. For instance, in regulatory decision making dilemmas may arise in which the goal of minimising risk runs counter to other commonly held priorities, like predictability of risk; in safety assessment using expert opinion, the commonly recognised risk of experts being "overconfident" may be less dangerous than their being under confident. 1 CSR TECHNICAL REPORT Contents IntroductionPredictions of reliability and safety through probabilistic modelling depend on the values of model parameters, e.g. component failure rates, which are often uncertain. The main application scenario that motivates our research involves decisions on accepting a software product for use in a safety critical application requiring low accident probability over the operational life of the system in which it is embedded. For instance, an explicit requirement in civil aviation is that "catastrophic failure conditions" be "so unlikely that they are not anticipated to occur during the entire operational life of all airplanes of one type" [16]. Nuclear power pro...

show abstract

Software Fault-Freeness and Reliability Predictions

Cited by 27 publications

References 15 publications

Assessing safety-critical systems from operational testing: A study on autonomous vehicles

Assessing safety-critical systems from operational testing: A study on autonomous vehicles

Software Diversity as a Measure for Reducing Development Risk

Bounds on survival probability given mean probability of failure per demand; and the paradoxical advantages of uncertainty

Contact Info

Product

Resources

About