Optimal Checkpointing Strategies for Iterative Applications

Abstract

This work provides an optimal checkpointing strategy to protect iterative applications from fail-stop errors. We consider a very general framework, where the application repeats the same execution pattern by executing consecutive iterations, and where each iteration is composed of several tasks. These tasks have different execution lengths and different checkpoint costs. Assume that there are n tasks and that task a i , where 0 ≤ i < n, has execution time t i and checkpoint cost C i . A naive strategy would checkpoint after each task. A strategy inspired by the Young/Daly formula would select the task a min with smallest checkpoint cost C min and would checkpoint after every p th instance of that task, leading to a checkpointing period P Y D = pT where T = n−1 i=0 a i is the time per iteration. One would choose the period so that P Y D = pT ≈ √ 2µC min to obey the Young/Daly formula, where µ is the application MTBF. Both the naive and Young/Daly strategies are suboptimal. Our main contribution is to show that the optimal checkpoint strategy is globally periodic, and to design a dynamic programming algorithm that computes the optimal checkpointing pattern. This pattern may well checkpoint many different tasks, and this across many different iterations. We show through simulations, both from synthetic and real-life application scenarios, that the optimal strategy significantly outperforms the naive and Young/Daly strategies.

Checkpointing à la Young/Daly: An Overview

“…One can approximate 𝐶 by the minimum, maximum, or average values of the checkpoint duration of all tasks. This is the heuristic proposed in [26], and its performance is shown satisfactory for a wide range of application scenarios.…”

“…As a side note, when the number of iterations is infinite (or very large in practice), it is shown in [26] that there exists an optimal checkpointing strategy that is periodic. It consists of a pattern of task outputs to checkpoint, where this pattern spans over a set of iterations of bounded size.…”

“…This pattern is repeated over and over throughout the execution. [26] also provides a dynamic programming algorithm, which is polynomial in the number of operations included in the outer loop to compute the optimal periodic checkpoint pattern. The complexity of the algorithm does not depend on the number of iterations of the outer loop.…”