Partial Redundancy in HPC Systems with Non-Uniform Node Reliabilities

Hussain, Zaeem; Znati, Taieb; Melhem, Rami

doi:10.1109/sc.2018.00047

Cited by 12 publications

(21 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A key contribution of this paper is a mathematical analysis of the restart strategy, with a closed-form formula for its optimal checkpointing period. We show that the optimal checkpointing period for the restart strategy has the order Θ(µ 2 3 ), instead of the Θ(µ 1 2 ) used in previous works for no-restart as an extension of the Young/Daly formula [11,20,25]. Hence, as the error rate increases, the optimal period becomes much longer than the value that has been used in all previous works (with no-restart).…”

Section: Introductionmentioning

confidence: 91%

“…The platform is subject to fail-stop errors, or failures, that interrupt the application. Similarly to previous work [17,20,25], for the mathematical analysis, we assume that errors are independent and identically distributed (IID), and that they strike each processor according to an exponential probability distribution exp(λ) with support [0, ∞), probability density function (PDF) f (t) = λe −λt and cumulative distribution function (CDF) F (T ) = P(X ≤ T ) = 1 − e −λT . We also introduce the reliability function G(T ) = 1 − F (T ) = e −λT .…”

Section: Modelmentioning

confidence: 99%

“…As discussed in [20,32], checkpoint time varies significantly depending upon the target application and the hardware capabilities. We will consider a time to checkpoint within two reasonable limits: 60s ≤ C ≤ 600s, following [25].…”

Section: Modelmentioning

confidence: 99%

“…Intuitively, since many failures are needed to interrupt the application, the checkpointing period should be much larger than without replication. Previous works [11,20,25] all use T no MTTI = √ 2M N C for the checkpointing period, where M N is the MTTI with N processors (instead of the MTBF µ N ).…”

Section: Introductionmentioning

confidence: 99%

“…One major contribution of this paper is to introduce a new approach that minimizes the overhead incurred by the checkpointrestart mechanism when coupled with replication. Previous works [11,20,25] use the no-restart strategy: if a processor was struck by a failure (but not its replica), then the processor remains failed (no recovery) until the whole application fails. Hence, there is a recovery only every M N seconds on average, whenever the application fails.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Replication is more efficient than you think

Benoît

Hérault

Fèvre

et al. 2019

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

This paper revisits replication coupled with checkpointing for failstop errors. Replication enables the application to survive many fail-stop errors, thereby allowing for longer checkpointing periods. Previously published works use replication with the no-restart strategy, which works as follows: (i) compute the application Mean Time To Interruption (MTTI) M as a function of the number of processor pairs and the individual processor Mean Time Between Failures (MTBF); (ii) use checkpointing period T no MTTI = √ 2MC à la Young/Daly, where C is the checkpoint duration; and (iii) never restart failed processors until the application crashes. We introduce the restart strategy where failed processors are restarted after each checkpoint. We compute the optimal checkpointing period T rs opt for this strategy, which is much larger than T no MTTI , thereby decreasing I/O pressure. We show through simulations that using T rs opt and the restart strategy, instead of T no MTTI and the usual no-restart strategy, significantly decreases the overhead induced by replication, in terms of both total execution time and energy consumption.

show abstract

Section: Introductionmentioning

confidence: 91%

Section: Modelmentioning

confidence: 99%

Section: Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Replication is more efficient than you think

Benoît

Hérault

Fèvre

et al. 2019

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

show abstract

Node failure resiliency for Uintah without checkpointing

Sahasrabudhe

Berzins

Schmidt

2019

Concurrency and Computation

View full text Add to dashboard Cite

The frequency of failures in upcoming exascale supercomputers may well be greater than at present due to many-core architectures if component failure rates remain unchanged. This potential increase in failure frequency coupled with I/O challenges at exascale may prove problematic for current resiliency approaches such as checkpoint restarting, although the use of fast intermediate memory may help. Algorithm-based fault tolerance (ABFT) using adaptive mesh refinement (AMR) is one resiliency approach used to address these challenges. For adaptive mesh codes, a coarse mesh version of the solution may be used to restore the fine mesh solution.This paper addresses the implementation of the ABFT approach within the Uintah software framework: both at a software level within Uintah and in the data reconstruction method used for the recovery of lost data. This method has two problems: inaccuracies introduced during the reconstruction propagate forward in time, and the physical consistency of variables, such as positivity or boundedness, may be violated during interpolation. These challenges can be addressed by the combination of two techniques: (1) a fault-tolerant message passing interface (MPI) implementation to recover from runtime node failures, and (2) high-order interpolation schemes to preserve the physical solution and reconstruct lost data. The approach considered here uses a ''limited essentially nonoscillatory'' (LENO) scheme along with AMR to rebuild the lost data without checkpointing using Uintah. Experiments were carried out using a fault-tolerant MPI-user-level failure mitigation to recover from runtime failure and LENO to recover data on patches belonging to failed ranks, while the simulation was continued to the end. Results show that this ABFT approach is up to 10× faster than the traditional checkpointing method. The new interpolation approach is more accurate than linear interpolation and not subject to the overshoots found in other interpolation methods.

show abstract

A survey of operating system support for persistent memory

Cai

Huang

2021

Front. Comput. Sci.

View full text Add to dashboard Cite

Partial Redundancy in HPC Systems with Non-Uniform Node Reliabilities

Cited by 12 publications

References 20 publications

Replication is more efficient than you think

Replication is more efficient than you think

Node failure resiliency for Uintah without checkpointing

A survey of operating system support for persistent memory

Contact Info

Product

Resources

About