Scalable Data Resilience for In-memory Data Staging

Duan, Shaohua; Subedi, Pradeep; Teranishi, Keita; Davis, Philip E.; Kolla, Hemanth; Gamell, Marc; Parashar, Manish

doi:10.1109/ipdps.2018.00021

Cited by 10 publications

(1 citation statement)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other areas of software fault tolerance focused on extreme scale and heterogeneous systems include resilience for task-parallel programming models [17,21], GPU snapshotting approaches [1], and the DataSpaces staging service [6] along with resilience strategies for multi-application staging [7]. Additionally, there has been recent focus on local checkpointing and rollback functionality to prevent a global rollback over the entire system [11,16,22].…”

Section: Related Workmentioning

confidence: 99%

Towards High Performance Resilience Using Performance Portable Abstractions

Morales

Teranishi

Nicolae

et al. 2021

Euro-Par 2021: Parallel Processing

Self Cite

View full text Add to dashboard Cite

In the drive towards Exascale, the extreme heterogeneity of supercomputers at all levels places a major development burden on HPC applications. To this end, performance portable abstractions such as those advocated by Kokkos, RAJA and HPX are becoming increasingly popular. At the same time, the unprecedented scalability requirements of such heterogeneous components means higher failure rates, motivating the need for resilience in systems and applications. Unfortunately, state-of-art resilience techniques based on checkpoint/restart are lagging behind performance portability efforts: users still need to capture consistent states manually, which introduces the need for fine-tuning and customization. In this paper we aim to close this gap by introducing a set of abstractions that make it easier for the application developers to reason about resilience. To this end, we extend the existing abstractions proposed by performance portability efforts towards resilience. By marking critical data structures that need to be checkpointed, one can enable an optimized runtime to automate checkpoint-restart using high performance and scalable asynchronously techniques. We illustrate the feasibility of our proposal using a prototype that combines the Kokkos runtime (HPC performance portability), with the VELOC runtime (large-scale low overhead checkpoint-restart). Our experimental results show negligible performance overhead compared compared with a manually tuned implementation of checkpoint-restart while requiring minimal changes in the application code.

show abstract