CRUM: Checkpoint-Restart Support for CUDA's Unified Memory

Abstract

Unified Virtual Memory (UVM) was recently introduced on recent NVIDIA GPUs. Through software and hardware support, UVM provides a coherent shared memory across the entire heterogeneous node, migrating data as appropriate. The older CUDA programming style is akin to older large-memory UNIX applications which used to directly load and unload memory segments. Newer CUDA programs have started taking advantage of UVM for the same reasons of superior programmability that UNIX applications long ago switched to assuming the presence of virtual memory. Therefore, checkpointing of UVM will become increasingly important, especially as NVIDIA CUDA continues to gain wider popularity: 87 of the top 500 supercomputers in the latest listings are GPU-accelerated, with a current trend of ten additional GPU-based supercomputers each year.A new scalable checkpointing mechanism, CRUM (Checkpoint-Restart for Unified Memory), is demonstrated for hybrid CUDA/MPI computations across multiple computer nodes. CRUM supports a fast, forked checkpointing, which mostly overlaps the CUDA computation with storage of the checkpoint image in stable storage. The runtime overhead of using CRUM is 6% on average, and the time for forked checkpointing is seen to be a factor of up to 40 times less than traditional, synchronous checkpointing.

Profiles of Upcoming HPC Applications and Their Impact on Reservation Strategies

“…Checkpointing GPU-enabled applications is difficult without a way to save the internal GPU state. Although some proxy-based approaches have been proposed [16], most actual implementations still rely on application-specific modification [43] which is not applicable to our study. Moreover using GPUs in Docker requires adhoc solutions such as NVIDIA-Docker that do not support checkpointing currently.…”

Predictive Reliability and Fault Management in Exascale Systems

“…The availability of effective and scalable checkpointing techniques for accelerators is thus essential for emerging exascale systems. Initial contributions [ 130 , 149 , 166 ] do not support features that are normally available in recent devices, such as NVIDIA Unified Virtual Addressing (UVA), as pointed out in Reference [ 74 ]. The decoupled CP U-GP U architecture poses additional technical challenges for effective application-wide checkpointing.…”

“…Other works, introduce some form of GPU checkpointing like HiAL-Ckpt [ 181 ], HeteroCheckpoint [ 102 ], and cudaCR [ 139 ], taking an application-specific approach to provide GPU-side checkpointing. Last, the CRUM framework presented in Reference [ 74 ], which also relies on a proxy-based approach along with new shadow page synchronization mechanisms, directly addresses the support for CUDA's unified virtual memory (UVM) available in the latest device generations, enabling fast asynchronous checkpointing for large-memory CUDA UVM applications and significantly reducing checkpointing overheads. While all the above contributions address GPU devices, FPGAs have emerged during recent years as an alternative for dedicated acceleration matching a few specific types of HPC workloads.…”

Efficient graphic processing unit implementation of the chemical-potential multiphase lattice Boltzmann method

“…Data transmission between the two GPUs in the kernel function is completed through the unified memory (Landaverde et al, 2014) and the peer to peer (P2P) transmission mode between the two GPUs shown in Figure 13 (Garg et al, 2018). These technologies hide data transmission between the GPUs.…”