Fault injection-based assessment of partial fault tolerance in stream processing applications

Jacques-Silva, Gabriela; Gedik, Buğra; Andrade, Henrique; Wu, Kun-Lung; Iyer, Ravishankar K.

doi:10.1145/2002259.2002292

Cited by 20 publications

(20 citation statements)

References 35 publications

(52 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First and foremost, we are working on a prototype which implements and applies the presented model to real systems, with the aim of fault diagnosis [30] and fault injection [31,32]. The core idea is to provide well-defined interfaces to keep the model in sync with the real target system.…”

Section: Resultsmentioning

confidence: 99%

“…Although not directly a part of this paper's contribution, we also briefly outline in the conclusions how the obtained model and fault taxonomy can be implemented and applied to real-life platforms with the goal of fault injection [31,32] and fault diagnosis [30], which is our focus for future work.…”

Section: Methodsmentioning

confidence: 99%

“…One solution is load shedding [1,18], where exactness/accuracy in the operator function is sacrificed for continuous availability by heuristically dropping (or delaying) events that are (deemed to be) of less importance. Partial fault tolerance under high load and the effects of bursty tuple loss have been intensively studied based on a fault injection framework in [31].…”

Section: Faults In Event Stream Processingmentioning

confidence: 99%

“…The authors of [31] have recently proposed a fault injection framework for assessing partial fault tolerance (PFT) of stream processing applications. Their work is tailored to high-frequency data streams and bursty tuple loss.…”

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

Deriving a unified fault taxonomy for event-based systems

Hummer

Inzinger

Leitner

et al. 2012

Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems

View full text Add to dashboard Cite

Dependability and fault-tolerance, which are key requirements for business-or safety-critical applications, require explicit knowledge of potential faults that may occur within a system. In contrast to other major research directions, the emerging field of distributed event-based systems is yet lacking a common understanding of faults. In this paper we take a step forward and study potential origins and effects of faults in such systems. Our work on a unified fault taxonomy follows a rigorous methodology. We first identify five core sub-areas in the broader field of event-based systems, and discuss commonalities and differences among them. Then we derive from the existing literature a coherent domain model, which accurately captures the specifics of the different areas. The domain model provides a holistic view and covers both structural and procedural aspects of event-based systems. Based on this model, we elaborate a detailed taxonomy of faults, in line with well-established fault dimensions from dependable and secure computing. The fault taxonomy forms the basis for a comprehensive discussion of fault instances across the five sub-areas of event processing.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Faults In Event Stream Processingmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Deriving a unified fault taxonomy for event-based systems

Hummer

Inzinger

Leitner

et al. 2012

Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems

View full text Add to dashboard Cite

show abstract

“…A fault injection-based approach is presented in [15] to evaluate the importance of the computation units to the output accuracy, which only considers independent failures. Zen [22] optimizes operator placement within clusters under a correlated failure model, which specifies the probability that a subset of the nodes fail together.…”

Section: Fault-tolerance In Spementioning

confidence: 99%

Tolerating correlated failures in Massively Parallel Stream Processing Engines

Zhou

2016

2016 IEEE 32nd International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

Abstract-Fault-tolerance techniques for stream processing engines can be categorized into passive and active approaches. A typical passive approach periodically checkpoints a processing task's runtime states and can recover a failed task by restoring its runtime state using its latest checkpoint. On the other hand, an active approach usually employs backup nodes to run replicated tasks. Upon failure, the active replica can take over the processing of the failed task with minimal latency. However, both approaches have their own inadequacies in Massively Parallel Stream Processing Engines (MPSPE). The passive approach incurs a long recovery latency especially when a number of correlated nodes fail simultaneously, while the active approach requires extra replication resources. In this paper, we propose a new faulttolerance framework, which is Passive and Partially Active (PPA). In a PPA scheme, the passive approach is applied to all tasks while only a selected set of tasks will be actively replicated. The number of actively replicated tasks depends on the available resources. If tasks without active replicas fail, tentative outputs will be generated before the completion of the recovery process. We also propose effective and efficient algorithms to optimize a partially active replication plan to maximize the quality of tentative outputs. We implemented PPA on top of Storm, an open-source MPSPE and conducted extensive experiments using both real and synthetic datasets to verify the effectiveness of our approach.

show abstract

A user‐assisted thread‐level vulnerability assessment tool

Topcuoglu

Tosun

2018

Concurrency and Computation

View full text Add to dashboard Cite

Summary The system reliability becomes a critical concern in modern architectures with the scale down of circuits. To deal with soft errors, the replication of system resources has been used at both hardware and software levels. Since the redundancy causes performance degradation, it is required to explore partial redundancy techniques that replicate the most vulnerable parts of the code. The redundancy level of user applications depends on user preferences and may be different for the users with different requirements. In this work, we propose a user‐assisted reliability assessment tool based on critical thread analysis for redundancy in parallel architectures. Our analysis evaluates the application threads of a parallel program by considering their criticality in the execution and selects the most critical thread or threads to be replicated. Moreover, we extend our analysis by exploring critical regions of individual threads and execute redundantly only those regions to reduce redundancy overhead. Our experimental evaluation indicates that the replication of the most critical thread improves the system reliability more (up to 10% for blackscholes application) than the replication of any other thread. The partial thread replication based on critical region analysis also reduces the vulnerability of the system by considering a fine‐grained approach.

show abstract

Fault injection-based assessment of partial fault tolerance in stream processing applications

Cited by 20 publications

References 35 publications

Deriving a unified fault taxonomy for event-based systems

Deriving a unified fault taxonomy for event-based systems

Tolerating correlated failures in Massively Parallel Stream Processing Engines

A user‐assisted thread‐level vulnerability assessment tool

Contact Info

Product

Resources

About