Performance model-directed data sieving for high-performance I/O

Abstract

Many scientific computing applications and engineering simulations exhibit noncontiguous I/O access patterns. Data sieving is an important technique to improve the performance of noncontiguous I/O accesses by combining small and noncontiguous requests into a large and contiguous request. It has been proven effective even though more data are potentially accessed than demanded. In this study, we propose a new data sieving approach namely performance model-directed data sieving, or PMD data sieving in short. It improves the existing data sieving approach from two aspects:(1) dynamically determines when it is beneficial to perform data sieving; and (2) dynamically determines how to perform data sieving if beneficial. It improves the performance of the existing data sieving approach considerably and reduces the memory consumption as verified by both theoretical analysis and experimental results. Given the importance of supporting noncontiguous accesses effectively and reducing the memory pressure in a large-scale system, the proposed PMD data sieving approach in this research holds a great promise and will have an impact on high-performance I/O systems.

A coarse‐grained page cache aware multivariate analytical model for the storage performance of a parallel file system

“…However, a proper accuracy evaluation of the proposed models is not presented in any of the above papers. Results in most of these papers 19,28,29,45 are restricted to show performance improvements obtained with optimizations driven by the proposed models. In this paper, we also propose a coarse-grained multivariate analytical model, which was developed over our preliminary work.…”

“…To identify the optimal distribution layout for a given application, He et al proposed an analytical cost model accounting for application, network, and storage characteristics to estimate access time in a heterogeneous PFS environment. Chen et al proposed an analytical model to estimate data access costs in a PFS and use it to dynamically determine when and how to perform data sieving to improve I/O performance.…”

“…7,8,23,24 Others leverage that knowledge and propose analytical and simulation models for PFSs. 19,[25][26][27][28][29] There are also works concerned with lower level parallel I/O and file assignment issues. 30 This paper reports our ongoing research work toward providing a comprehensive understanding about the performance of PFSs.…”