TWINS: Server Access Coordination in the I/O Forwarding Layer

Abstract

This paper presents a study of I/O scheduling techniques applied to the I/O forwarding layer. In high-performance computing environments, applications rely on parallel file systems (PFS) to obtain good I/O performance even when handling large amounts of data. To alleviate the concurrency caused by thousands of nodes accessing a significantly smaller number of PFS servers, intermediate I/O nodes are typically applied between processing nodes and the file system. Each intermediate node forwards requests from multiple clients to the system, a setup which gives this component the opportunity to perform optimizations like I/O scheduling. We evaluate scheduling techniques that improve spatiality and request size of the access patterns. We show they are only partially effective because the access pattern is not the main factor for read performance in the I/O forwarding layer. A new scheduling algorithm, TWINS, is presented to coordinate the access of intermediate I/O nodes to the data servers. Our proposal decreases concurrency at the data servers, a factor previously proven to negatively affect performance. The proposed algorithm is able to improve read performance from shared files by up to 28% over other scheduling algorithms and by up to 50% over not forwarding I/O.

Detecting I/O Access Patterns of HPC Workloads at Runtime

“…TWINS [6] is an I/O scheduling algorithm designed for the I/O forwarding layer of large-scale clusters and supercomputers. It seeks to coordinate the I/O nodes' accesses to the shared PFS servers to mitigate contention.…”

“…The values on the xaxis represent different TWINS window sizes. Further details are available in previous work [6], [28].…”

“…In general, optimization techniques typically provide improvements for particular system configurations and access patterns, but not for all of them. Furthermore, they often rely on the right selection of parameters, as demonstrated for request scheduling at different levels [5], [6]. Therefore, achieving the best results proposed by an optimization technique often relies on correctly applying them to the proper workload, and configuring it accordingly.…”

Cross‐layer coordination in the I/O software stack of extreme‐scale systems

“…TWINS is an I/O scheduling technique at the I/O forwarding layer. It uses the time windows and coordinates accesses form I/O nodes so that at each time window they focus on one of the storage nodes.…”

On server-side file access pattern matching

“…Grenoble Alpes patterns, but not for all of them. Moreover, they often depend on the right choice of parameters, as demonstrated for request scheduling at different levels [5], [6]. In this situation, finding ways of adapting the optimizations is key to achieving good performance.…”