Predicting I/O Performance in HPC Using Artificial Neural Networks

Abstract

The prediction of file access times is an important part for the modeling of supercomputer's storage systems. These models can be used to develop analysis tools which support the users at integrating efficient I/O behavior. In this paper, we analyze and predict the access times of a Lustre file system from the client perspective. For this purpose, we measured file access times in various test series and develop different models for predicting access times. The evaluation shows that in models utilizing artificial neural networks the average prediciton error is about 30% smaller than in linear models. One phenomenon in the distribution of file access times is of particular interest: File accesses with identical parameters show several typical access times. The typical access times usually differ by orders of magnitude and can be explained with a different processing of the file accesses in the storage system-an alternative I/O path. We investigate a method to automatically determine the alternative I/O path and quantify the significance of knowledge about the internal processing. It is shown that the prediction error is improved significantly with this approach.

I/O-signature-based feature analysis and classification of high-performance computing applications

“…The second is papers that predict performance using various ML techniques based on I/O characteristics analysis [13,18,23,26,28]. In [23], the authors evaluate predictors of I/O performance using machine learning with artificial neural networks.…”

“…The second is papers that predict performance using various ML techniques based on I/O characteristics analysis [13,18,23,26,28]. In [23], the authors evaluate predictors of I/O performance using machine learning with artificial neural networks. For this, they measured file access times in various test series and develop different models for predicting access times.…”