Distributed Checkpointing on Clusters with Dynamic Striping and Staggering

Jin, Hai; Hwang, Kai

doi:10.1007/3-540-36184-7_4

Cited by 4 publications

(4 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[35] presents diskless checkpointing. Other interesting schemes are based on: ReedSolomon coding [8], cutoff and compression to reduce checkpoint sizes [23], checkpointing on clouds [33], reducing I/O bottlenecks [25], and performant checkpoints to PFS [7].…”

Section: Checkpointing Protocolsmentioning

confidence: 99%

“…Logging-based protocols Pessimistic [18] Causal [16] Receiver-based [17] Non-blocking [11] Uncoordinated [18,20,20,41] Coordinated [25] Comm.-induced Dep. graph [9] Checkp.…”

Section: Other Schemesmentioning

confidence: 99%

“…Multi-level [8,30,38] Page-based [40] Hash-based [4] Diskless [35] Library level OS level Stable storage [7,25] Adaptive [4] Compiler enhanced [10] Application level…”

Section: Size Of Checkpointsmentioning

confidence: 99%

See 2 more Smart Citations

Fault tolerance for remote memory access programming models

Besta

Hoefler

2014

Proceedings of the 23rd International Symposium on High-Performance Parallel and Distributed Computing

View full text Add to dashboard Cite

Remote Memory Access (RMA) is an emerging mechanism for programming high-performance computers and datacenters. However, little work exists on resilience schemes for RMA-based applications and systems. In this paper we analyze fault tolerance for RMA and show that it is fundamentally different from resilience mechanisms targeting the message passing (MP) model. We design a model for reasoning about fault tolerance for RMA, addressing both flat and hierarchical hardware. We use this model to construct several highly-scalable mechanisms that provide efficient low-overhead in-memory checkpointing, transparent logging of remote memory accesses, and a scheme for transparent recovery of failed processes. Our protocols take into account diminishing amounts of memory per core, one of the major features of future exascale machines. The implementation of our fault-tolerance scheme entails negligible additional overheads. Our reliability model shows that inmemory checkpointing and logging provide high resilience. This study enables highly-scalable resilience mechanisms for RMA and fills a research gap between fault tolerance and emerging RMA programming models.

show abstract

Section: Checkpointing Protocolsmentioning

confidence: 99%

“…Logging-based protocols Pessimistic [18] Causal [16] Receiver-based [17] Non-blocking [11] Uncoordinated [18,20,20,41] Coordinated [25] Comm.-induced Dep. graph [9] Checkp.…”

Section: Other Schemesmentioning

confidence: 99%

See 1 more Smart Citation

Fault tolerance for remote memory access programming models

Besta

Hoefler

2014

Proceedings of the 23rd International Symposium on High-Performance Parallel and Distributed Computing

View full text Add to dashboard Cite

show abstract

“…However, contention for stable storage is still a problem when several processes take checkpoints simultaneously. This can significantly impact the checkpointing overhead and extend the total execution time of the distributed computation [30,31,12,5,20]. Contention for stable storage can be mitigated by staggering the checkpoints [27].…”

Section: Introductionmentioning

confidence: 99%