Predicting how badly "good" software can behave

Voas, E.; Charron, F.; McGraw, Gary; Miller, Keith; Friedman, Michael

doi:10.1109/52.595959

Cited by 92 publications

(45 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, error propagation is a useful property in software testing, as it helps to reveal state corruption due to faults by propagating such faults to the interfaces of the system Miller, 1994a, 1995;Voas et al, 1997]. Therefore, for components with high error propagation probability, testing is more likely to reveal faults in the code, if they are present.…”

Section: Related Workmentioning

confidence: 99%

“…The authors introduce the term failure tolerance to mean that the system is tolerant to failures of 3 rd party software. In [Voas et al, 1997] the authors further speculate that a similar technique would be most useful for an OS setting, since system software consists of a multitude of interacting components.…”

Section: Related Workmentioning

confidence: 99%

“…al. presents EPA (Extended Propagation Analysis) [Voas et al, , 1997Voas, 1997a] which identifies code locations which might violate the safety requirements of the system if faults occur in these locations. The authors introduce the term failure tolerance to mean that the system is tolerant to failures of 3 rd party software.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Robustness Evaluation of Operating Systems

Johansson

2008

Information Assurance

View full text Add to dashboard Cite

SummaryThe premise behind this thesis is the observation that Operating Systems (OS), being the foundation behind operations of computing systems, are complex entities and also subject to failures. Consequently, when they do fail, the impact is the loss of system service and the applications running thereon. While a multitude of sources for OS failures exist, device drivers are often identified as a prominent cause behind failures.In order to characterize the impact of driver failures, at both the OS and application levels, this thesis develops a framework for error propagation-based robustness profiling for an OS. The framework is first developed conceptually and then experimentally validated on a real OS, namely Windows CE .Net. The choice of Windows CE is driven by its representativeness for a multitude of OS's, as well as the ability to customize the OS components for particular needs.For experimental validation, fault injection is a prominent technique that can be used to simulate faults (or errors) in the system by inserting synthetic ones and study their effect. Three key questions with such a technique are where, what and when to inject faults. This thesis shows how injecting errors at the interface between drivers and the OS can be very effective in evaluating the effects driver faults can have.To quantify the OS's robustness, this thesis defines a series of error propagation measures, specifically tailored for device drivers. These measures allow for quantifying and comparing both individual services and device drivers on their susceptibility and diffusing abilities.This thesis compares three contemporary error models on their suitability for robustness evaluation. The classical bit-flip model is found to identify a higher number of severe failures in the system. It also identifies failures for more services than both other models, data type and fuzzing. However, its main drawback is that it requires substantially more injections than the other two. Fuzzing, even though not giving rise to as many failures is able to find new additional services with severe failures.A careful study of the injections performed with the bit-flip model shows that only a few bits are generally useful for identifying new services with robustness weaknesses. Consequently, a new composite model is proposed, combining the most effective bits of the bit-flip model with the fuzzing model's ability to identify new services, giving rise to new model without loss of important information and at the same time incurring a moderate number of injections.To answer the question of when to inject an error this thesis proposes a novel model of a driver's usage profile, focusing on high-level operations being carried out. It guides the injection of errors to instances when the driver is carrying out specific operations. Results from extensive fault injection experiments show that more service vulnerabilities can be discovered. Furthermore, a priori profiling of the drivers can show how effective the proposed approach will be. vi

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Robustness Evaluation of Operating Systems

Johansson

2008

Information Assurance

View full text Add to dashboard Cite

show abstract

“…For an extensive overview of mutation-based testing and fault models, the reader should refer to [16]. More studies about the various aspects of software fault injection and fault models can be found in [12,13,17,24].…”

Section: Software Fmeamentioning

confidence: 99%

Model Checking-based Software-FMEA: Assessment of Fault Tolerance and Error Detection Mechanisms

Molnár

Majzik

2017

Period. Polytech. Elec. Eng. Comp. Sci.

View full text Add to dashboard Cite

Failure Mode and Effects Analysis (FMEA) is a systematic IntroductionThe risk of failure is one of the main concerns of safetycritical systems. Certification requires the systematic analysis of potential failures, their causes and effects, and the evaluation of risk mitigation techniques used to reduce the chance and the severity of system-level failures.One of the first systematic techniques for failure analysis was Failure Mode and Effect Analysis (FMEA) [2]. FMEA is often the first step in reliability analysis, as it collects the potential failure modes of subsystems, their root causes and their effects on the whole system. Together with criticality analysis (often treated as part of FMEA, sometimes emphasized by the term FMECA), the output of FMEA serves as the basis of other qualitative and quantitative analyses, as well as design decisions regarding risk mitigation techniques.FMEA is usually applied at the hardware and communication level, where it requires a qualified analyst to collect postulated component failures and identify their effects on other components and the system level. In case of software (SW-FMEA), failure modes originate in different types of programming faults, commonly referred to as bugs. Due to the complex nature of software and the many types of potential bugs, it is much harder to collect failure modes and deduce their potential effects, so an automated mechanism is desired. This paper presents a way of automated SW-FMEA with the use of executable software models. Assuming a set of predefined fault types (programming faults) and a specification of safe behavior at the system level, the proposed approach applies model checking to systematically generate execution traces leading from fault activations to states that violate the specification of safe behavior (system-level failures). These traces can be used to understand and demonstrate fault propagation through the system and also as test sequences to reveal actual faults in the final product.Our approach improves existing model checking-based SW-FMEA by optimizing fault activations and using monitor components instead of a formal specification. The optimization helps in scaling the solution to apply it on more complex systems and situations, whereas using a monitor instead of formal

show abstract

“…There are a number of programs and tools for efficient fault injection [2,7,10,22,24,26,28,31,35,43,45,49,55,56]. There are efforts to model fault propagation and effect [5,7,12,17,19,22,23,27,28,30,32,36,38,39,40,41,44,45,47,50,52]. Coverage is a popular topic, and the meaning of coverage can vary from simple detection to complete system recovery and reconfiguration [1,2,3,4,5,10,11,12,15,16,17,18,20,21,27,29,…”

Section: Literature Surveymentioning

confidence: 99%

Designing fault-injection experiments for the reliability of embedded systems

White

2012

2012 IEEE/AIAA 31st Digital Avionics Systems Conference (DASC)

View full text Add to dashboard Cite

This paper considers the long-standing problem of conducting fault-injections experiments to establish the ultra-reliability of embedded systems. There have been extensive efforts in fault injection, and this paper offers a partial summary of the efforts, but these previous efforts have focused on realism and efficiency. Fault injections have been used to examine diagnostics and to test algorithms, but the literature does not contain any framework that says how to conduct fault-injection experiments to establish ultra-reliability. A solution to this problem integrates field-data, arguments-from-design, and fault-injection into a seamless whole. The solution in this paper is to derive a model reduction theorem for a class of semi-Markov models suitable for describing ultra-reliable embedded systems. The derivation shows that a tight upper bound on the probability of system failure can be obtained using only the means of system-recovery times, thus reducing the experimental effort to estimating a reasonable number of easily-observed parameters. The paper includes an example of a system subject to both permanent and transient faults. There is a discussion of integrating fault-injection with fielddata and arguments-from-design.

show abstract

Predicting how badly "good" software can behave

Cited by 92 publications

References 7 publications

Robustness Evaluation of Operating Systems

Robustness Evaluation of Operating Systems

Model Checking-based Software-FMEA: Assessment of Fault Tolerance and Error Detection Mechanisms

Designing fault-injection experiments for the reliability of embedded systems

Contact Info

Product

Resources

About