Optimal Resilience Patterns to Cope with Fail-Stop and Silent Errors

Abstract

International audienceThis work focuses on resilience techniques at extreme scale. Many papers deal with fail-stop errors. Many others deal with silent errors (or silent data corruptions). But very few papers deal with fail-stop and silent errors simultaneously. However, HPC applications will obviously have to cope with both error sources. This paper presents a unified framework and optimal algorithmic solutions to this double challenge. Silent errors are handled via verification mechanisms (either partially or fully accurate) and in-memory checkpoints. Fail-stop errors are processed via disk checkpoints. All verification and checkpoint types are combined into computational patterns. We provide a unified model, and a full characterization of the optimal pattern. Our results nicely extend several published solutions and demonstrate how to make use of different techniques to solve the double threat of fail-stop and silent errors. Extensive simulations based on real data confirm the accuracy of the model, and show that patterns that combine all resilience mechanisms are required to provide acceptable overheads

Coping with silent and fail-stop errors at scale by combining replication and checkpointing

“…which is consistent with the results obtained in [2,6,7], provided that a reliable silent error detector is available. However, as mentioned previously, such a detector is only known in some application-specific domains.…”

Combining Checkpointing and Replication for Reliable Execution of Linear Workflows

“…Di et al [12] analyzed a two-level computational pattern, and proved that equal-length checkpointing segments constitute the optimal solution. Benoit et al [3] relied on disk checkpoints to cope with fail-stop failures and used memory checkpoints coupled with error detectors to handle silent data corruptions. They derived first-order approximation formulas for the optimal pattern length as well as the number of memory checkpoints between two disk checkpoints.…”

Detection of Silent Data Corruptions in Smoothed Particle Hydrodynamics Simulations

“…Checkpointing with rollback recovery [17,23] is the de-facto general-purpose recovery technique in high-performance computing. Finding the optimal checkpointing interval [7,19,21,49] or the optimal recovery method for SPH codes is beyond the scope of this paper.…”