Coordinated checkpoint versus message log for fault tolerant MPI

Bouteiller, Aurélien; Lemarinier, Pierre; Krawezik,; Capello,

doi:10.1109/clustr.2003.1253321

Cited by 61 publications

(67 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The most common fault-tolerance technique used in high performance computing is checkpointrollback-recovery [6,7,5,2]. A large body of work has studied periodic coordinated checkpointing for a single divisible application.…”

Section: Related Workmentioning

confidence: 99%

“…Too infrequent checkpoints lead to wasteful re-computation when a failure occurs, but too frequent checkpoints lead to overhead during failure-free periods of the application execution. Checkpointing can happen in a coordinated or uncoordinated manner, and the advantages and drawbacks of both approaches are well-documented [5]. Checkpointing can be agnostic to the application, in which case full address space images are saved as checkpoints [6,7].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Checkpointing Strategies for Scheduling Computational Workflows

Aupy

Benoît

Casanova

et al. 2016

IJNC

View full text Add to dashboard Cite

We study the scheduling of computational workflows on compute resources that experience exponentially distributed failures. When a failure occurs, rollback and recovery is used to resume the execution from the last checkpointed state. The scheduling problem is to minimize the expected execution time by deciding in which order to execute the tasks in the workflow and deciding for each task whether to checkpoint it or not after it completes. We give a polynomialtime optimal algorithm for fork DAGs (Directed Acyclic Graphs) and show that the problem is NP-complete with join DAGs. We also investigate the complexity of the simple case in which no task is checkpointed. Our main result is a polynomial-time algorithm to compute the expected execution time of a workflow, with a given task execution order and specified to-be-checkpointed tasks. Using this algorithm as a basis, we propose several heuristics for solving the scheduling problem. We evaluate these heuristics for representative workflow configurations.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Checkpointing Strategies for Scheduling Computational Workflows

Aupy

Benoît

Casanova

et al. 2016

IJNC

View full text Add to dashboard Cite

show abstract

“…Moreover the overhead induced during failure-free execution decreases the performances in not very faulty environments, such as clusters [23]. Furthermore, it can lead to the domino effect [24]: a process that rollbacks and that need a message to be replayed, asks another process to rollback. This process does, and asks another one to do so, etc.…”

Section: Number Of Rollbacksmentioning

confidence: 99%

Performance comparison of hierarchical checkpoint protocols grid computing

Ndiaye¹,

Sens²,

Thiaré³

2012

IJIMAI

View full text Add to dashboard Cite

-46- Abstract -Grid infrastructure is a large set of nodes geographically distributed and connected by a communication. In this context, fault tolerance is a necessity imposed by the distribution that poses a number of problems related to the heterogeneity of hardware, operating systems, networks, middleware, applications, the dynamic resource, the scalability, the lack of common memory, the lack of a common clock, the asynchronous communication between processes. To improve the robustness of supercomputing applications in the presence of failures, many techniques have been developed to provide resistance to these faults of the system. Fault tolerance is intended to allow the system to provide service as specified in spite of occurrences of faults. It appears as an indispensable element in distributed systems. To meet this need, several techniques have been proposed in the literature. We will study the protocols based on rollback recovery. These protocols are classified into two categories: coordinated checkpointing and rollback protocols and log-based independent checkpointing protocols or message logging protocols. However, the performance of a protocol depends on the characteristics of the system, network and applications running. Faced with the constraints of large-scale environments, many of algorithms of the literature showed inadequate. Given an application environment and a system, it is not easy to identify the recovery protocol that is most appropriate for a cluster or hierarchical environment, like grid computing. While some protocols have been used successfully in small scale, they are not suitable for use in large scale. Hence there is a need to implement these protocols in a hierarchical fashion to compare their performance in grid computing. In this paper, we propose hierarchical version of four well-known protocols. We have implemented and compare the performance of these protocols in clusters and grid computing using the Omnet++ simulator.

show abstract

“…Several implementations of message logging protocols over Message Passing Interface (MPI) library have been developed by Cappello et al [2,3,9]. The well known MPICH-V included the three different protocols (pessimistic, optimistic and causal) into the same library and it was one of the first fault tolerant distributions of MPI.…”

Section: Introductionmentioning

confidence: 99%