How Much SSD Is Useful for Resilience in Supercomputers

Abstract

We consider the use of non-volatile memories in the form of burst buffers for resilience in supercomputers. Their cost and limited lifetime demand effective use and appropriate provisioning. We develop an analytic model for the behavior of workloads on systems with burst buffers, and use it to explore questions of cost-effective provisioning, and missiondirected allocation of burst-buffer (SSD) lifetime.First, our results show that system efficiency can be increased by as much as 14% by considering a global perspective (workload mix, job size) for SSD lifetime allocation. Second, with size-based and system-efficiency based lifetime allocation, large jobs suffer as much as 40% job efficiency loss; job-efficiency based allocation must increase their allocations by 50% to eliminate this disparity. Finally, further results suggest that underprovisioning SSD lifetime (only 10-20% of the "optimum" as defined by per-job requirements without resource constraint) is sufficient to produce 90% system efficiency at failure rates three times that of current systems.

Optimizing checkpoint data placement with guaranteed burst buffer endurance in large-scale hierarchical storage systems

“…In order to better utilize SSD devices under scientific I/O workloads, [14] presents a checkpoint interval optimization model for large-scale scientific applications. This model is essentially same as those developed by [26,27,28] whose objective is to maximize the computational efficiency of HPC systems, but it also takes the constraint of burst buffer capacity into consideration.…”

“…When model AUBNP is used, all the checkpoint data is written to the burst buffer without limit while no checkpoint is written to the PFS directly, and the checkpoint interval is determined through adaptive algorithm named "lazy checkpoint" (see section 3.2). When model SLBNP is used, limited checkpoint data is written to the burst buffer while no checkpoint is written to the PFS directly, and the checkpoint interval is determined through the optimization model proposed in [14] (see section 3.3). SLBUP and ALBUP are two models proposed in this paper, which all limit the checkpoint data written to the burst buffer while leverage the PFS to keep the checkpoint frequency from decreasing too much (see section 4 and 5 for more details).…”

“…Besides frequently replacing worn-out SSD devices, another possible solution would be reducing the amount of data written to the burst buffer. In [14], the authors proposed a checkpoint interval optimization model for large-scale scientific applications which takes the constraint of burst buffer capacity into consideration. In such model, SSD-based burst buffers of supercomputers are used to absorb all checkpoint data of the scientific applications.…”

“…We use ∆t wr ckpt,i to denote the time spent on writing one checkpoint of the i-th job to the storage system. [26,27,28,14]. If the failure rate per compute node in the HPC system is λ and the number of compute nodes occupied by the i-th job is n i , the total execution time of the i-th job can be denoted as:…”

Exploring versioned distributed arrays for resilience in scientific applications

“…A wide array of checkpoint-restart research has explored techniques to efficiently apply checkpointing (Daly, 2006;Young, 1974) and improve its performance (Antypas et al, 2014;Bautista-Gomez et al, 2011;Cappello et al, 2011;Fang and Chien, 2015;Moody et al, 2010). Notably, recent advances that exploit high bandwidth non-volatile memories both reduce checkpoint cost dramatically, and because their efficiency reduces the optimal checkpoint interval, can do ''micro-checkpointing'' (fast checkpointing with interval of seconds), reducing the work lost per detected error or process failure.…”