MLBS: Transparent Data Caching in Hierarchical Storage for Out-of-Core HPC Applications

Abstract

Out-of-core simulation systems produce and/or consume a massive amount of data that cannot fit on a single compute node memory and that usually needs to be read and/or written back and forth during computation. I/O data movement may thus represent a bottleneck in large-scale simulations. To increase I/O bandwidth, high-end supercomputers are equipped with hierarchical storage subsystems such as node-local and remote-shared NVMe and SSD-based Burst Buffers. Advanced caching systems have recently been developed to efficiently utilize the multi-layered nature of the new storage hierarchy. Utilization of software components results in more efficient data accesses, at the cost of reduced computation kernel performance and limited numbers of simultaneous applications that can utilize the additional storage layers. We introduce MultiLayered Buffer Storage (MLBS), a data object container that provides novel methods for caching and prefetching data in out-of-core scientific applications to perform asynchronously expensive I/O operations on systems equipped with hierarchical storage. The main idea consists in decoupling I/O operations from computational phases using dedicated hardware resources to perform expensive context switches. MLBS monitors I/O traffic in each storage layer allowing fair utilization of shared resources while controlling the impact on kernels' performance. By continually prefetching up and down across all hardware layers of the memory/storage subsystems, MLBS transforms the original I/O-bound behavior of evaluated applications and shifts it closer to a memorybound regime. Our evaluation on a Cray XC40 system for a representative I/O-bound application, seismic inversion, shows that MLBS outperforms state-of-the-art filesystems, i.e., Lustre, Data Elevator and DataWarp by 6.06X, 2.23X, and 1.90X, respectively.

Towards Efficient I/O Pipelines Using Accumulated Compression

“…2) On-the-fly Individual Checkpoint Compression: In this approach, the checkpoints are compressed one at a time at their source, i.e., the GPU HBM in our case, by the checkpointing runtime. This approach is widely used for accelerating data transfer for out-of-core stencil computations [22], reverse-mode adjoint computations [23], and reducing datastream intensity from scientific equipment, e.g., Advanced Photon source [24]. Therefore, we consider this approach as representative of state-of-the-art GPU-compression-enabled data movement techniques.…”

Towards Efficient Cache Allocation for High-Frequency Checkpointing

“…System-level checkpointing libraries such as NVCR [24] and CheCUDA [23], transparently record and replay the memory based CUDA APIs for checkpointing and restoring. Approaches such as CheckFreq [30], GPUsnapshot [31] and Multi-layered Buffered System [32] also exploit heterogeneous storage tiers. However, none of these approaches consider short-running jobs, for which the impact of initialization overheads is non-negligible.…”

“…One such popular scenario is the use of checkpointing for the purpose of revisiting previous states in order to advance a computation. For example, the adjoint state method is an efficient numerical method to compute the gradient that is widely employed by automatic differentiation (AD) tools [2], [3] and used in a variety of scientific applications: climate and ocean modeling [4], multi-physics [5], seismic imaging in the oil industry [6], etc. Deep learning (DL) techniques are also based on AD and often paired with stochastic gradient descent [7].…”

Towards Efficient I/O Scheduling for Collaborative Multi-Level Checkpointing

“…Systemlevel checkpoint-restart libraries, e.g., CheCUDA [16] and NVCR [15] transparently record and replay all the memorybased API calls. Efforts such as MLBS [21] and Check-Freq [22] leverage multi-level memory subsystem starting from GPU memory to minimize checkpoint time. However, none of these approaches analyze the imbalance of checkpoint sizes across multiple devices, nor do they leverage peer-to-peer transfers to reduce the checkpointing overheads.…”