I/O Optimizations Based on Workload Characteristics for Parallel File Systems

Abstract

Parallel file systems usually provide a unified storage solution, which fails to meet specific application needs. In this paper, we propose an extended file handle scheme to address this problem. It allows the file systems to specify optimizations for individual file or directory based on workload characteristics. One case study shows that our proposed approach improves the aggregate throughput of large files and small files by up to 5% and 30%, respectively. To further improve the access performance of small files in parallel file systems, we also propose a new metadata-based small file optimization method. The experimental results show that the aggregate throughput of small files can be effectively improved through our method.

Efficient Approaches for Metadata Prefetching and Data Placement in Wide-Area Computing

“…Consequently, different prefetch requirements cannot be met. Regarding metadata access, existing prefetching policies can be improved in the following three aspects: 1) most non-frequency-mining-based prefetching approaches rely heavily on simple patterns (e.g., temporal locality, sequentiality, and loop references) and do not fully exploit semantic correlations between files [10]; 2) considerable frequency-mining-based prefetching work cannot easily capture file associations imposed by users, particularly when the historical access information is not sufficient to learn associations [11]; and 3) choosing a reasonable prefetch-ing policy in concurrent workload scenarios is not straightforward, as the data of different applications are simultaneously handled by different processes [12].…”