I/O acceleration with pattern detection

He, Jie; Bent, John; Torres, Aaron; Grider, Gary; Gibson, Garth A.; Maltzahn, Carlos; Sun, Xian-He

doi:10.1145/2462902.2462909

Cited by 26 publications

(13 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, higher performance is achieved when accessing sequentially positioned portions of a file in large requests as opposed to accessing small sparse portions. For this reason, many optimization techniques ( [5], [6], [7]) have been proposed aiming at adapting the access pattern to achieve the best performance from the file system.…”

Section: Introductionmentioning

confidence: 99%

Collective I/O Performance on the Santos Dumont Supercomputer

Carneiro

Bez

Boito

et al. 2018

2018 26th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP)

View full text Add to dashboard Cite

The historical gap between processing and data access speeds causes many applications to spend a large portion of their execution on I/O operations. From the point of view of a large-scale, expensive, supercomputer, it is important to ensure applications achieve the best I/O performance to promote an efficient usage of the machine. In this paper, we evaluate the I/O infrastructure of the Santos Dumont supercomputer, the largest one from Latin America. More specifically, we investigate the performance of collective I/O operations. By conducting an analysis of a scientific application that uses the machine, we identify large performance differences between the available MPI implementations. We then further study the observed phenomenon using the BT-IO and IOR benchmarks, in addition to a custom microbenchmark. We conclude that the customized MPI implementation by Bull (used by more than 20% of the jobs) presents the worst performance for small collective write operations. Our results are being used to help the Santos Dumont users to achieve the best performance for their applications. Additionally, by investigating the observed phenomenon, we provide information to help improve future MPI-IO collective write implementations.

show abstract

Section: Introductionmentioning

confidence: 99%

Collective I/O Performance on the Santos Dumont Supercomputer

Carneiro

Bez

Boito

et al. 2018

2018 26th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP)

View full text Add to dashboard Cite

show abstract

“…To put it from another way, the approach of data prefetching focuses on determining what to prefetch, it cares about how to forecast the future possible read operation, which is critical to the effectiveness of data prefetching. There are more sophisticated approaches for predictive prefetching, also named automatic prefetching schemes that utilize HHM or neural nets or other predictive algorithms to forecast I/O operations by using I/O access patterns of the application [5,6,[29][30][31][32][33]. These systems employ specific algorithms to predict the application's future I/O access request based on past I/O tracks.…”

Section: Related Workmentioning

confidence: 99%

Server‐side prefetching in distributed file systems

Liao

2014

Concurrency and Computation

View full text Add to dashboard Cite

This paper presents a proactive data prefetching mechanism on storage servers for distributed file systems to achieve better input/output (I/O) performance. This mechanism requires keeping tracks of what blocks were accessed and the information about client I/O requests to predict which blocks will be accessed in the future. Except for showing a linear access tendency, I/O access operations on blocks also reveal a random or similarrandom tendency but with certain access patterns. Thus, to address the prediction for future possible read operations, a linear regression algorithm and a chaotic prediction algorithm have been proposed to forecast the future access requests according to different access patterns. As a result, by employing the information, that is, offsets and request sizes, about the predicted future read requests on disk blocks, the storage servers in our newly presented mechanism can prefetch block data more precisely and then send it to associated client file system proactively. The experimental results illustrate that our proposed mechanism is able to shorten read latency to a great extent and then contribute to better read throughput significantly in certain application contexts. 310 J. LIAO 44. Ruemmler C, Wilkes J. A trace-driven analysis of disk working set sizes. Technical Report HPL-OSR-93-23, HP Labs, 1993. 45. Narayan S, Chandy JA. Trace based analysis of file system effects on disk I/O.

show abstract

“…Besides, J. He et al [23] have explored and classified patterns of I/O within applications, thereby allowing powerful I/O optimization strategies including pattern-aware prefetching to enhance I/O performance.…”

Section: I/o Optimization By Using Either Logical I/o Access Informatmentioning

confidence: 99%

Dynamic Stripe Management Mechanism in Distributed File Systems

Liao

Xiao

Liu

et al. 2014

Advanced Information Systems Engineering

View full text Add to dashboard Cite

This paper presents a novel mechanism to dynamically resize and re-distribute stripes on the storage servers in distributed file systems. To put this mechanism to work, the information about logical I/O access on the client side is piggybacked to physical I/O access on the storage server side, for building the relationship between the logical I/O access and physical I/O access. Moreover, this newly presented mechanism supports varying size of stripes on the storage servers to obtain finer concurrency granularity on accessing to data stripes. As a result, the mapping relationship can be utilized to direct stripe re-sizing and re-distributing on the storage servers dynamically for better system performance. Experimental results show that this stripe management mechanism can reduce I/O response time and boost I/O data throughput significantly for applications with complicated access patterns.

show abstract

I/O acceleration with pattern detection

Cited by 26 publications

References 32 publications

Collective I/O Performance on the Santos Dumont Supercomputer

Collective I/O Performance on the Santos Dumont Supercomputer

Server‐side prefetching in distributed file systems

Dynamic Stripe Management Mechanism in Distributed File Systems

Contact Info

Product

Resources

About