MPI/FT: A Model-Based Approach to Low-Overhead Fault Tolerant Message-Passing Middleware

Batchu, R.; Dandass, Yoginder S.; Skjellum, Anthony; Murali, Beddhu

doi:10.1023/b:clus.0000039491.64560.8a

Cited by 37 publications

(18 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Software-based solutions detect the anomalies in the behavior of a system's data variables or control flow attributes to determine the presence of a fault. Heartbeat monitoring is used for liveness checking of MPI processes, which enables detection of imminent failure of the MPI communicator [6].…”

Section: Fault Treatment Patternmentioning

confidence: 99%

Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale

Hukerikar

Engelmann

2017

JSFI

View full text Add to dashboard Cite

Reliability is a serious concern for future extreme-scale high-performance computing (HPC) systems. Projections based on the current generation of HPC systems and technology roadmaps suggest the prevalence of very high fault rates in future systems. While the HPC community has developed various resilience solutions, application-level techniques as well as system-based solutions, the solution space remains fragmented. There are no formal methods and metrics to integrate the various HPC resilience techniques into composite solutions, nor are there methods to holistically evaluate the adequacy and efficacy of such solutions in terms of their protection coverage, and their performance & power efficiency characteristics. Additionally, few of the current approaches are portable to newer architectures and software environments that will be deployed on future systems. In this paper, we develop a structured approach to the design, evaluation and optimization of HPC resilience using the concept of design patterns. A design pattern is a general repeatable solution to a commonly occurring problem. We identify the problems caused by various types of faults, errors and failures in HPC systems and the techniques used to deal with these events. Each well-known solution that addresses a specific HPC resilience challenge is described in the form of a pattern. We develop a complete catalog of such resilience design patterns, which may be used by system architects, system software and tools developers, application programmers, as well as users and operators as essential building blocks when designing and deploying resilience solutions. We also develop a design framework that enhances a designer's understanding the opportunities for integrating multiple patterns across layers of the system stack and the important constraints during implementation of the individual patterns. It is also useful for defining mechanisms and interfaces to coordinate flexible fault management across hardware and software components. The resilience patterns and the design framework also enable exploration and evaluation of design alternatives and support optimization of the cost-benefit trade-offs among performance, protection coverage, and power consumption of resilience solutions. The overall goal of this work is to establish a systematic methodology for the design and evaluation of resilience technologies in extreme-scale HPC systems that keep scientific applications running to a correct solution in a timely and cost-efficient manner despite frequent faults, errors, and failures of various types.

show abstract

Section: Fault Treatment Patternmentioning

confidence: 99%

Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale

Hukerikar

Engelmann

2017

JSFI

View full text Add to dashboard Cite

show abstract

“…These ABFT techniques typically require a fault-tolerant message passing environment. There have been a number of these resilient message passing libraries based on MPI, including; FT-MPI [22,17], AMPI [9], MPI/FT [3], and C 3 [8]. The differences between these libraries is beyond the scope of this work, but each of these libraries allows for an application to continue operating in the presence of faults, possibly in a degraded mode, and it is left up to the application to ensure the result is correct.…”

Section: Fault Tolerant Userspace Librariesmentioning

confidence: 99%

Evaluating operating system vulnerability to memory errors

Ferreira

Pedretti

Brightwell

et al. 2012

Proceedings of the 2nd International Workshop on Runtime and Operating Systems for Supercomputers

View full text Add to dashboard Cite

“…As a result, they can get away with not completely handling nondeterministic events. Other projects such as FT-MPI [43] and MPI/FT [44] have extended MPI to implement process replicas on MPI applications for hard faults. PLR applies replicas for transient fault tolerance on general-purpose multicore machines.…”

Section: Related Workmentioning

confidence: 99%

PLR: A Software Approach to Transient Fault Tolerance for Multicore Architectures

Shye¹,

Blomstedt

Moseley

et al. 2009

IEEE Trans. Dependable and Secure Comput.

118

View full text Add to dashboard Cite

Abstract-Transient faults are emerging as a critical concern in the reliability of general-purpose microprocessors. As architectural trends point toward multicore designs, there is substantial interest in adapting such parallel hardware resources for transient fault tolerance. This paper presents process-level redundancy (PLR), a software technique for transient fault tolerance, which leverages multiple cores for low overhead. PLR creates a set of redundant processes per application process and systematically compares the processes to guarantee correct execution. Redundancy at the process level allows the operating system to freely schedule the processes across all available hardware resources. PLR uses a software-centric approach to transient fault tolerance, which shifts the focus from ensuring correct hardware execution to ensuring correct software execution. As a result, many benign faults that do not propagate to affect program correctness can be safely ignored. A real prototype is presented that is designed to be transparent to the application and can run on general-purpose single-threaded programs without modifications to the program, operating system, or underlying hardware. The system is evaluated for fault coverage and performance on a four-way SMP machine and provides improved performance over existing software transient fault tolerance techniques with a 16.9 percent overhead for fault detection on a set of optimized SPEC2000 binaries.

show abstract

MPI/FT: A Model-Based Approach to Low-Overhead Fault Tolerant Message-Passing Middleware

Cited by 37 publications

References 18 publications

Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale

Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale

Evaluating operating system vulnerability to memory errors

PLR: A Software Approach to Transient Fault Tolerance for Multicore Architectures

Contact Info

Product

Resources

About