Evaluating I/O characteristics and methods for storing structured scientific data

2012 IEEE 26th International Parallel and Distributed Processing Symposium

2012

Abstract-The parallel data accesses inherent to large-scale data-intensive scientific computing requires that data servers handle very high I/O concurrency. Concurrent requests from different processes or programs to hard disk can cause disk head thrashing between different disk regions, resulting in unacceptably low I/O performance. Current storage systems either rely on the disk scheduler at each data server, or use SSD as storage, to minimize this negative performance impact. However, the ability of the scheduler to alleviate this problem by scheduling requests in memory is limited by concerns such as long disk access times, and potential loss of dirty data with system failure. Meanwhile, SSD is too expensive to be widely used as the major storage device in the HPC environment.We propose iTransformer, a scheme that employs a small SSD to schedule requests for the data on disk. Being less space constrained than with more expensive DRAM, iTransformer can buffer larger amounts of dirty data before writing it back to the disk, or prefetch a larger volume of data in a batch into the SSD. In both cases high disk efficiency can be maintained even for concurrent requests. Furthermore, the scheme allows the scheduling of requests in the background to hide the cost of random disk access behind serving process requests. Finally, as a non-volatile memory, concerns about quantity of dirty data are obviated. We have implemented iTransformer in the Linux kernel and tested it on a large cluster running PVFS2. Our experiments show that iTransformer can improve the I/O throughput of the cluster by 35% on average for MPI/IO benchmarks of various data access patterns.

Section: The Hpio Benchmarkmentioning

confidence: 99%

iTransformer: Using SSD to Improve Disk Scheduling for High-performance I/O

2012 IEEE 26th International Parallel and Distributed Processing Symposium

2012

“…hpio is a program designed by Northwestern University and Sandia National Laboratories to systematically evaluate I/O performance using a diverse set access patterns [7]. This benchmark program can generate differing data access patterns by changing three parameters: region count, region spacing, and region size.…”

Section: A the Benchmarksmentioning

confidence: 99%

IOrchestrator: Improving the Performance of Multi-node I/O Systems via Inter-Server Coordination

2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis

2010

Abstract-A cluster of data servers and a parallel file system are often used to provide high-throughput I/O service to parallel programs running on a compute cluster. To exploit I/O parallelism parallel file systems stripe file data across the data servers. While this practice is effective in serving asynchronous requests, it may break individual program's spatial locality, which can seriously degrade I/O performance when the data servers concurrently serve synchronous requests from multiple I/O-intensive programs.In this paper we propose a scheme, IOrchestrator, to improve I/O performance of multi-node storage systems by orchestrating I/O services among programs when such inter-data-server coordination is dynamically determined to be cost effective. We have implemented IOrchestrator in the PVFS2 parallel file system. Our experiments with representative parallel benchmarks show that IOrchestrator can significantly improve I/O performanceby up to a factor of 2.5-delivered by a cluster of data servers servicing concurrently-running parallel programs. Notably, we have not observed any scenarios in which the use of IOrchestrator causes substantial performance degradation.

“…In addition to the demonstration program we used in Section 1 to exhibit the resonance scenario, we used five wellknown benchmark programs for the evaluation: coll perf from the MPICH2 software package, mpi-io-test from the PVFS2 software package, ior-mpi-io from the ASCI Purple benchmark suite developed at Lawrence Livermore National Laboratory [10], noncontig from the Parallel I/O Benchmarking Consortium [20] at Argonne National Laboratory to test I/O characteristics with noncontiguous file access [19], and hpio, designed by Northwestern University and Sandia National Laboratories, to systematically evaluate performance with a diverse set of I/O access patterns [9,2]. All presented measurements represent arithmetic means of three runs.…”

Section: Performance Evaluation and Analysismentioning

confidence: 99%

“…The benchmark HPIO can generate various data access patterns by changing three parameters: region count, region spacing, and region size [2]. In our experiment, we set region count to 4096, region spacing to 0, and vary region size from 2KB to 64KB.…”

Section: Benchmark Hpiomentioning

confidence: 99%

Making resonance a common case: A high-performance implementation of collective I/O on parallel file systems

2009 IEEE International Symposium on Parallel &Amp; Distributed Processing

2009

Collective I/O is a widely used technique to improve I/O performance in parallel computing. It can be implemented as a client-based or server-based scheme. The client-based implementation is more widely adopted in MPI-IO software such as ROMIO because of its independence from the storage system configuration and its greater portability. However, existing implementations of client-side collective I/O do not take into account the actual pattern of file striping over multiple I/O nodes in the storage system. This can cause a significant number of requests for non-sequential data at I/O nodes, substantially degrading I/O performance.Investigating the surprisingly high I/O throughput achieved when there is an accidental match between a particular request pattern and the data striping pattern on the I/O nodes, we reveal the resonance phenomenon as the cause. Exploiting readily available information on data striping from the metadata server in popular file systems such as PVFS2 and Lustre, we design a new collective I/O implementation technique, resonant I/O, that makes resonance a common case. Resonant I/O rearranges requests from multiple MPI processes according to the presumed data layout on the disks of I/O nodes so that non-sequential access of disk data can be turned into sequential access, significantly improving I/O performance without compromising the independence of a client-based implementation. We have implemented our design in ROMIO. Our experimental results on a small-and medium-scale cluster show that the scheme can increase I/O throughput for some commonly used parallel I/O benchmarks such as mpi-io-test and ior-mpi-io over the existing implementation of ROMIO by up to 157%, with no scenario demonstrating significantly decreased performance.