Survey of Storage Systems for High-Performance Computing

Abstract

In current supercomputers, storage is typically provided by parallel distributed file systems for hot data and tape archives for cold data. These file systems are often compatible with local file systems due to their use of the POSIX interface and semantics, which eases development and debugging because applications can easily run both on workstations and supercomputers. There is a wide variety of file systems to choose from, each tuned for different use cases and implementing different optimizations. However, the overall application performance is often held back by I/O bottlenecks due to insufficient performance of file systems or I/O libraries for highly parallel workloads. Performance problems are dealt with using novel storage hardware technologies as well as alternative I/O semantics and interfaces. These approaches have to be integrated into the storage stack seamlessly to make them convenient to use. Upcoming storage systems abandon the traditional POSIX interface and semantics in favor of alternative concepts such as object and key-value storage; moreover, they heavily rely on technologies such as NVM and burst buffers to improve performance. Additional tiers of storage hardware will increase the importance of hierarchical storage management. Many of these changes will be disruptive and require application developers to rethink their approaches to data management and I/O. A thorough understanding of today's storage infrastructures, including their strengths and weaknesses, is crucially important for designing and implementing scalable storage systems suitable for demands of exascale computing.

Lightweight Integration of a Data Cache for Opportunistic Usage of HPC Resources in HEP Workflows

“…The aim was to demonstrate that decent results can already be achieved by setting up a caching instance on an existing system landscape. The setup most likely would benefit from dedicated hardware, e.g., SSD storage for the cache space (due to the better latency, throughput, and input/output operations per second with respect to HDD storage [16]) or more RAM for the proxy server (especially for high numbers of parallel file requests). However, the use and optimization of additional hardware was out of scope for this study since we wanted to concentrate on commodity hardware which can be found in any data center.…”

Predicting file lifetimes for data placement in multi-tiered storage systems for HPC

“…The use of tape is the final piece of the puzzle, as it is used to archive the data. Tapes have a latency of around 20 seconds or more [24]. Data stored on tapes does not consume any power which makes it an ideal solution for long term storage.…”

“…Since no single storage technology is capable of fulfilling both those roles at a reasonable cost [13], we need to use a hierarchy of different storage technologies. The combined use of Flash-based drives, magnetic drives (HDD) and tapes is standard [24] in HPC systems. Memory class storage such as 3dX-point are expected to join the aforementioned hierarchy in production to partially fill the gap between DRAM and Flash [17] [5].…”

“…The standard architecture used in HPC is a three tier hierarchy [19] composed of clusters of Flash-based drives, magnetic drives and tapes each grouped together in storage nodes and managed by a parallel distributed file system or an object store such as DAOS [22]. Because the storage capacity available on the high performance tier is often an order of magnitude lower than that of the other tiers [24], the fastest tier of this hierarchy should only contain a subset of files for the time period during which they require such a performance.…”

SLRL

“…Technologies such as NVM [16] [5] and SSDs [4] offer high performance but their high cost in currency per gigabyte and thus low capacity prevent them from being used as the only form of storage (in most cases). Instead, companies are using HDD [25] for their moderate cost and higher capacity, with a smaller amount of faster SSD or NVM being used as a cache or as a complementary storage (horizontal integration) [4]. These storage devices are eventually completed with tapes [20], offering long-term low-cost but high-latency technologies as a third storage tier.…”