iBridge: Improving Unaligned Parallel File Access with Solid-State Drives

Zhang, Xuechen; Liu, Ke; Davis, Kei; Jiang, Song

doi:10.1109/ipdps.2013.21

Cited by 37 publications

(16 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The hybrid SSD+HDD approach is also used by Zhang et al [120]. They apply SSDs to store "fragments", which are the initial and final portions of files that are not stripe size aligned.…”

Section: Caching and Prefetchingmentioning

confidence: 99%

A Checkpoint of Research on Parallel I/O for High-Performance Computing

et al. 2018

View full text Add to dashboard Cite

We present a comprehensive survey on parallel I/O in the high performance computing (HPC) context. This is an important field for HPC because of the historic gap between processing power and storage latencies, which causes applications performance to be impaired when accessing or generating large amounts of data. As the available processing power and amount of data increase, I/O remains a central issue for the scientific community. In this survey, we focus on a traditional I/O stack, with a POSIX parallel file system. We present background concepts everyone could benefit from. Moreover, through the comprehensive study of publications from the most important conferences and journals in a five-year time window, we discuss the state of the art of I/O optimization approaches, access pattern extraction techniques, and performance modeling, in addition to general aspects of parallel I/O research. Through this approach, we aim at identifying the general characteristics of the field and the main current and future research topics.

show abstract

“…The hybrid SSD+HDD approach is also used by Zhang et al [120]. They apply SSDs to store "fragments", which are the initial and final portions of files that are not stripe size aligned.…”

Section: Caching and Prefetchingmentioning

confidence: 99%

A Checkpoint of Research on Parallel I/O for High-Performance Computing

et al. 2018

View full text Add to dashboard Cite

show abstract

“…iTransformer [7] applies SSDs to buffer extended HDDs scheduling queue for optimization. iBridge [8] introduces a scheme which uses SSDs to host fragments, thereby alleviates the speed gap between access fragments and large files. LiU [9] studies how to place different partitions of a file to HDDs and SSDs to relieve I/O latency of HDDs.…”

Section: Related Workmentioning

confidence: 99%

Exploring Memory Hierarchy to Improve Scientific Data Read Performance

Zhang

Tang

Zou

et al. 2015

2015 IEEE International Conference on Cluster Computing

View full text Add to dashboard Cite

Improving read performance is one of the major challenges with speeding up scientific data analytic applications. Utilizing the memory hierarchy is one major line of researches to address the read performance bottleneck. Related methods usually combine solide-state-drives(SSDs) with dynamic random-access memory(DRAM) and/or parallel file system(PFS) to mitigate the speed and space gap between DRAM and PFS. However, these methods are unable to handle key performance issues plaguing SSDs, namely read contention that may cause up to 50% performance reduction. In this paper, we propose a framework that exploits the memory hierarchy resource to address the read contention issues involved with SSDs. The framework employs a general purpose online read algorithm that able to detect and utilize memory hierarchy resource to relieve the problem. To maintain a near optimal operating environment for SSDs, the framework is able to orchastrate data chunks across different memory layers to facilitate the read algorithm. Compared to existing tools, our framework achieves up to 50% read performance improvement when tested on datasets from real-world scientific simulations.

show abstract

“…SieveStore [24] captures the most popular blocks and places them onto an SSD. iTransformer [25] and iBridge [26] redirect the most random requests to the SSD to achieve cost-effective storage acceleration. SSD-based hybrid storage is another costeffective method to make full use of the potential of SSDs.…”

Section: Data Placement In Ssd-based Storage Systemmentioning

confidence: 99%

A cost-aware region-level data placement scheme for hybrid parallel I/O systems

Sun

Feng

et al. 2013

2013 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

Parallel I/O systems represent the most commonly used engineering solution to mitigate the performance mismatch between CPU and disk performance; however, parallel I/O systems are application dependent and may not work well for certain data access requests. New emerging solid state drives (SSD) are able to deliver better performance but incur a high monetary cost. While SSDs cannot always replace HDDs, the hybrid SSD-HDD approach uniquely addresses common performance issues in parallel I/O systems. The performance of hybrid SSD-HDD architecture depends on the utilization of the SSD and scheduling of data placement. In this paper, we propose a cost-aware regionlevel (CARL) data placement scheme for hybrid parallel I/O systems. CARL divides large files into several small regions, calculates the region costs according to the data access patterns, and selectively places regions with high access costs onto the SSDbased file servers. We have implemented CARL under MPI-IO and the PVFS2 parallel file system environment. Experimental results of representative benchmarks show that CARL is both feasible and able to improve I/O performance significantly.

show abstract

iBridge: Improving Unaligned Parallel File Access with Solid-State Drives

Cited by 37 publications

References 17 publications

A Checkpoint of Research on Parallel I/O for High-Performance Computing

A Checkpoint of Research on Parallel I/O for High-Performance Computing

Exploring Memory Hierarchy to Improve Scientific Data Read Performance

A cost-aware region-level data placement scheme for hybrid parallel I/O systems

Contact Info

Product

Resources

About