Emulation of Software Faults: A Field Data Study and a Practical Approach

Duraes, Joao; Madeira, Henrique

doi:10.1109/tse.2006.113

Cited by 233 publications

(253 citation statements)

References 32 publications

Supporting

Mentioning

226

Contrasting

Order By: Relevance

“…In [19] three main types of faults are identified: missing, wrong and extraneous code constructs. The classification was applied to diff and patch files of open source projects.…”

Section: Related Workmentioning

confidence: 99%

Deriving a unified fault taxonomy for event-based systems

Hummer

Inzinger

Leitner

et al. 2012

Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems

View full text Add to dashboard Cite

Dependability and fault-tolerance, which are key requirements for business-or safety-critical applications, require explicit knowledge of potential faults that may occur within a system. In contrast to other major research directions, the emerging field of distributed event-based systems is yet lacking a common understanding of faults. In this paper we take a step forward and study potential origins and effects of faults in such systems. Our work on a unified fault taxonomy follows a rigorous methodology. We first identify five core sub-areas in the broader field of event-based systems, and discuss commonalities and differences among them. Then we derive from the existing literature a coherent domain model, which accurately captures the specifics of the different areas. The domain model provides a holistic view and covers both structural and procedural aspects of event-based systems. Based on this model, we elaborate a detailed taxonomy of faults, in line with well-established fault dimensions from dependable and secure computing. The fault taxonomy forms the basis for a comprehensive discussion of fault instances across the five sub-areas of event processing.

show abstract

“…In [19] three main types of faults are identified: missing, wrong and extraneous code constructs. The classification was applied to diff and patch files of open source projects.…”

Section: Related Workmentioning

confidence: 99%

Deriving a unified fault taxonomy for event-based systems

Hummer

Inzinger

Leitner

et al. 2012

Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems

View full text Add to dashboard Cite

show abstract

“…Each defect can be classified to belong to one of these types. These types have later been used to build libraries for injection of artificial faults in systems, for instance [Christmansson and Chillarege, 1996;Christmansson et al, 1998;Durães and Madeira, 2006].…”

Section: Software Relatedmentioning

confidence: 99%

“…G-SWFIT (Generic Software Fault injection Technique) uses software mutations to inject software faults into the binary of a target program [Durães and Madeira, 2006]. A field study of real faults was used to generate mutations.…”

Section: Fault Injectionmentioning

confidence: 99%

“…This process is started automatically at boot time and simply tries to reboot the system after a specific timeout has triggered (currently four minutes). To define such a timeout is common practice for fault injection [Arlat et al, 1990;Durães and Madeira, 2006]. If this also fails a manual hardware reset is required by the evaluator.…”

Section: Experiments Managermentioning

confidence: 99%

“…used a five grade scale for their robustness benchmark [Siewiorek et al, 1993]. Many fault injection tools have used similar scales as well, like MAFALDA , NFTAPE [Gu et al, 2003] and others, for instance [Chillarege and Bowen, 1989;Barton et al, 1990;Marsden and Fabre, 2001;Durães and Madeira, 2006]. As a representative example of failure modes defined from an application perspective, the CRASH severity scale presented in [Koopman et al, 1997] is shown in Table 5.1.…”

Section: Failure Mode Analysismentioning

confidence: 99%

See 2 more Smart Citations

Robustness Evaluation of Operating Systems

Johansson

2008

Information Assurance

View full text Add to dashboard Cite

SummaryThe premise behind this thesis is the observation that Operating Systems (OS), being the foundation behind operations of computing systems, are complex entities and also subject to failures. Consequently, when they do fail, the impact is the loss of system service and the applications running thereon. While a multitude of sources for OS failures exist, device drivers are often identified as a prominent cause behind failures.In order to characterize the impact of driver failures, at both the OS and application levels, this thesis develops a framework for error propagation-based robustness profiling for an OS. The framework is first developed conceptually and then experimentally validated on a real OS, namely Windows CE .Net. The choice of Windows CE is driven by its representativeness for a multitude of OS's, as well as the ability to customize the OS components for particular needs.For experimental validation, fault injection is a prominent technique that can be used to simulate faults (or errors) in the system by inserting synthetic ones and study their effect. Three key questions with such a technique are where, what and when to inject faults. This thesis shows how injecting errors at the interface between drivers and the OS can be very effective in evaluating the effects driver faults can have.To quantify the OS's robustness, this thesis defines a series of error propagation measures, specifically tailored for device drivers. These measures allow for quantifying and comparing both individual services and device drivers on their susceptibility and diffusing abilities.This thesis compares three contemporary error models on their suitability for robustness evaluation. The classical bit-flip model is found to identify a higher number of severe failures in the system. It also identifies failures for more services than both other models, data type and fuzzing. However, its main drawback is that it requires substantially more injections than the other two. Fuzzing, even though not giving rise to as many failures is able to find new additional services with severe failures.A careful study of the injections performed with the bit-flip model shows that only a few bits are generally useful for identifying new services with robustness weaknesses. Consequently, a new composite model is proposed, combining the most effective bits of the bit-flip model with the fuzzing model's ability to identify new services, giving rise to new model without loss of important information and at the same time incurring a moderate number of injections.To answer the question of when to inject an error this thesis proposes a novel model of a driver's usage profile, focusing on high-level operations being carried out. It guides the injection of errors to instances when the driver is carrying out specific operations. Results from extensive fault injection experiments show that more service vulnerabilities can be discovered. Furthermore, a priori profiling of the drivers can show how effective the proposed approach will be. vi

show abstract

Benchmarking the Impact of Faulty Drivers: Application to the Linux Kernel

Kanoun

Spainhower

2008

Dependability Benchmarking for Computer Systems

View full text Add to dashboard Cite

Dependability concerns, encompassing robustness assessment, are essential questions to answer before a developer can make the decision whether to integrate off-the-shelf (OTS) components into a dependable system. Here, and in what follows, robustness is understood as the degree to which a system operates correctly in the presence of exceptional inputs or stressful environmental conditions, in compliance with the generic definition (dependability with respect to external faults) given in [Aviz ienis et al. 2004].From a cost-effectiveness viewpoint, operating systems and kernels are privileged OTS components as candidates for integration into a system. However, integrators are often reluctant to make such a move without obtaining a deeper knowledge and understanding about such a component beyond functional issues, in particular with respect to its failure modes and its behavior in the presence of faults. Due to the opacity that is often attached to the commercial offer and to the difficulty and significant cost associated with the availability of the source code, the Open Source option, for which access to the source code is granted, is progressively making its way as an attractive and promising alternative. Also, results of many studies have shown that Open Source solutions did not exhibit significantly more critical failure modes and in some cases they were even found to demonstrate behaviors superior to commercial options [Koopman and DeVale 1999, Marsden et al. 2002, Vieira & Madeira 2003; see also Chapter 5 for the latter. In this chapter, we will simply denote such components (either commercial or Open Source) as OTS components.In the past years, several experimental studies have addressed this important issue from different perspectives [Koopman and DeVale 1999 (see also Chapter 11), . This has also led to the proposal of tentative dependability benchmarking approaches, aimed at characterizing the robustness of computer systems Dependability Benchmarking for Computer Systems.

show abstract

Emulation of Software Faults: A Field Data Study and a Practical Approach

Cited by 233 publications

References 32 publications

Deriving a unified fault taxonomy for event-based systems

Deriving a unified fault taxonomy for event-based systems

Robustness Evaluation of Operating Systems

Benchmarking the Impact of Faulty Drivers: Application to the Linux Kernel

Contact Info

Product

Resources

About