PPFS: A Scale-out Distributed File System for Post-petascale Systems

Abstract

Abstract:The fusion of the research field of high-performance computing (HPC) with that of big data, which has become known as the field of extreme big data, is problematic in that file creation in storage systems such as distributed file systems is not optimized. That is, the large workload leads to simultaneous creations of many files by many processes when creating checkpoints. The need to improve the file creation processes prompted us to design a scale-out distributed file system for post-petascale systems named PPFS. PPFS consists of PPMDS, which is a scale-out distributed metadata server, and PPOSS, which is a scalable distributed storage server for flash storage. The high file creation performance of PPMDS was achieved by using a key-value store for metadata storage and non-blocking distributed transactions to update multiple entries simultaneously. PPOSS depends on PPOST, which is an object storage system that manages the underlying low-level storage, such as Fusion IO ioDrive, a flash device connected through PCI express supporting OpenNVM. The high file creation performance was attained by implementing the PPFS prototype using file creation optimization, termed bulk creation, to reduce the amount of communication between PPMDS and PPOSS. And, to enhance the I/O performance of PPOSS when the client process and PPOSS run on the same node, PPOSS accesses a local storage device directly. The prototype implementation of PPFS with a further file creation optimization called object prefetching achieves 138,000 Operations Per Second for file creation when using five metadata servers and 128 client processes, thereby exceeding the performance of IndexFS by 2.52 times. With local access optimization, PPOSS reached its limit at a block size of 16 KiB, which is an improvement of 1.5 times compared to before optimization. Furthermore, this evaluation indicates that PPFS has a good scalability on file creation and IO performance, that is required for post-petascale systems.

Scalable Distributed Metadata Server Based on Nonblocking Transactions

“…To cope with the issue, this paper proposed a scalable metadata server system, PPMDS. PPMDS is motivated from the Gfarm [Tatebe et al 2010] distributed file system and is already integrated with an object storage [Takatsu et al 2016] and forms a file system called PPFS [Takatsu et al 2017].…”

“…PPFS [Takatsu et al 2017] is a distributed file system that utilizes the PP-MDS [Hiraga et al 2018] for file system metadata management. It introduces the design of the PPFS file system using the PPMDS and PPOST object storagebased file servers [Takatsu et al 2016].…”

Process Data Infrastructure and Data Services

“…Using a similar approach, Takatsu et al present PPFS (Post-Petascale File System), a DFS optimized for high file creation workloads. In their paper they argue that modern DFSs are not optimized for high file creation workloads, and that for exascale computing this can turn out to be a serious performance bottleneck [42]. They have evaluated their system against IndexFS (2014), a middleware for file systems such as HDFS and Lustre aiming to improve metadata performance [43].…”