BatchFS: Scaling the File System Control Plane with Client-Funded Metadata Servers

Abstract

Abstract-Parallel file systems are often characterized by a layered architecture that decouples metadata management from I/O operations, allowing file systems to facilitate fast concurrent access to file contents. However, metadata intensive workloads are still likely to bottleneck at the file system control plane due to namespace synchronization, which taxes application performance through lock contention on directories, transaction serialization, and RPC overheads. In this paper, we propose a client-driven file system metadata architecture, BatchFS, that is optimized for noninteractive, or batch, workloads. To avoid metadata bottlenecks, BatchFS features a relaxed consistency model marked by lazy namespace synchronization and optimistic metadata verification. Capable of executing namespace operations on client-provisioned resources without contacting any metadata server, BatchFS clients are able to delay namespace synchronization until synchronization is really needed. Our goal in this vision paper is to handle these delayed operations securely and efficiently with metadata verification and bulk insertion. Preliminary experiments demonstrate that our client-funded metadata architecture outperforms a traditional synchronous file system by orders of magnitude.

PPFS: A Scale-out Distributed File System for Post-petascale Systems

“…PPMDS uses non-blocking transactions to resolve this issue. BatchFS [12] is an extension of IndexFS to reduce the RPC (Remote Procedure Call) overhead by using a relaxed consistency model. DeltaFS [13] is a serverless file system designed for exascale computing.…”

DeltaFS

“…Because access to a global namespace typically requires centralized serializable coordination with dedicated resources and is all too often a performance bottleneck, designing a file system without a global namespace and without any dedicated metadata servers should, in principle, enable the system to grow to much larger scales in a more cost-effective way. In BatchFS [16], our prior extension to IndexFS, each file system client is able to instantiate a private namespace in a snapshot of the global namespace and obtain capabilities to pre-execute metadata operations directly in that private namespace. At a later point, a client would choose to submit changes to its private namespace to the global namespace to merge updates through a single bulk insertion operation [16].…”

“…In BatchFS [16], our prior extension to IndexFS, each file system client is able to instantiate a private namespace in a snapshot of the global namespace and obtain capabilities to pre-execute metadata operations directly in that private namespace. At a later point, a client would choose to submit changes to its private namespace to the global namespace to merge updates through a single bulk insertion operation [16]. We showed that this asynchronous mechanism can afford a batch job the ability to efficiently execute a huge set of concurrent metadata operations with only a small interaction with centralized metadata servers, which is particularly useful for achieving high-performance checkpointing [16][17][18][19].…”

“…DeltaFS represents namespace metadata as ordered keyvalue pairs that are managed by LevelDB [25] using LSMtrees implemented on storage as SSTables (Sorted String Tables). Each SSTable stores a range of immutable and indexed key-value entries and serves as the physical format for metadata migration and aggregation [15,16]. SSTables are ordered with entries in newer tables superseding entries in older ones.…”

“…Serverless file systems [6,39] are traditionally characterized by a set of symmetric file servers that are each capable of serving the entire file system namespace. This architecture is reused by both BatchFS [16] and DeltaFS to enable flexible metadata migration and service allocation, and is also extended by DeltaFS to be literally "serverless".…”