Cooperative Write-Behind Data Buffering for MPI I/O

Liao, Wei-keng; Coloma, Kenin; Choudhary, Alok; Ward, Lee

doi:10.1007/11557265_17

Cited by 15 publications

(10 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the regular distribution studied before, we can describe these relationships in mathematical form (6).…”

Section: Automatic Datatype Generation For Iec Techniquementioning

confidence: 99%

“…Several researchers have contributed with optimizations of MPI-IO data operations: data sieving [16], non-contiguous access [18], collective caching [7], cooperating write-behind buffering [6], integrated collective I/O and cooperative caching [4]. In another approach [20], the optimal setting for MPI-IO file hints is automatically computed.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A collective I/O implementation based on inspector–executor paradigm

et al. 2008

View full text Add to dashboard Cite

In this paper, we present a novel multiple phase I/O collective technique for generic block-cyclic distributions. The I/O technique is divided into two stages: inspector and executor. During the inspector stage, the communication pattern is computed and the required datatypes are automatically generated. This information is used during the executor stage in performing the communication and file accesses. The two stages are decoupled, so that for repetitive file access patterns, the computations from the inspector stage can be performed once and reused several times by the executor. This strategy allows to amortize the inspector cost over several I/O operations. In this paper, we evaluate the performance of multiple phase I/O collective technique and we compare it with other state of the art approaches. Experimental results show that for small access granularities, our method outperforms in the large majority of cases other parallel I/O optimizations techniques.

show abstract

“…For the regular distribution studied before, we can describe these relationships in mathematical form (6).…”

Section: Automatic Datatype Generation For Iec Techniquementioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

A collective I/O implementation based on inspector–executor paradigm

et al. 2008

View full text Add to dashboard Cite

show abstract

“…Several researchers have contributed with optimizations of MPI-IO data operations: data sieving [10], non-contiguous access [11], collective caching [12], cooperating write-behind buffering [13], integrated collective I/O, and cooperative caching [14] are some major examples.…”

Section: I/o Performance Optimization Techniquesmentioning

confidence: 99%

Dynamic-CoMPI: dynamic optimization techniques for MPI parallel applications

et al. 2010

View full text Add to dashboard Cite

This work presents an optimization of MPI communications, called Dynamic-CoMPI, which uses two techniques in order to reduce the impact of communications and non-contiguous I/O requests in parallel applications. These techniques are independent of the application and complementaries to each other. The first technique is an optimization of the Two-Phase collective I/O technique from ROMIO, called Locality aware strategy for Two-Phase I/O (LA-Two-Phase I/O). In order to increase the locality of the file accesses, LA-Two-Phase I/O employs the Linear Assignment Problem (LAP) for finding an optimal I/O data communication schedule. The main purpose of this technique is the reduction of the number of communications involved in the I/O collective operation. The second technique, calledAdaptive-CoMPI, is based on run-time compression of MPI messages exchanged by applications. Both techniques can be applied on every application, because both of them are transparent for the users. Dynamic-CoMPI has been validated by using several MPI benchmarks and real HPC applications. The results show that, for many of the considered scenarios, important reductions in the execution time are achieved by reducing the size and the number of the messages. Additional benefits of our approach are the reduction of the total communication time and the network contention, thus enhancing, not only performance, but also scalability.

show abstract

“…Several researchers have contributed with optimizations of MPI-IO data operations: data sieving [10], non-contiguous access [16], collective caching [17], cooperating write-behind buffering [18], integrated collective I/O and cooperative caching [14].…”

Section: Related Workmentioning

confidence: 99%

Data Locality Aware Strategy for Two-Phase Collective I/O

Filgueira

Singh

Pichel

et al. 2008

High Performance Computing for Computational Science - VECPAR 2008

View full text Add to dashboard Cite

Abstract. This paper presents Locality-Aware Two-Phase (LATP) I/O, an optimization of the Two-Phase collective I/O technique from ROMIO, the most popular MPI-IO implementation. In order to increase the locality of the file accesses, LATP employs the Linear Assignment Problem (LAP) for finding an optimal distribution of data to processes, an aspect that is not considered in the original technique. This assignment is based on the local data that each process stores and has as main purpose the reduction of the number of communication involved in the I/O collective operation and, therefore, the improvement of the global execution time. Compared with Two-Phase I/O, LATP I/O obtains important improvements in most of the considered scenarios.

show abstract

Cooperative Write-Behind Data Buffering for MPI I/O

Cited by 15 publications

References 7 publications

A collective I/O implementation based on inspector–executor paradigm

A collective I/O implementation based on inspector–executor paradigm

Dynamic-CoMPI: dynamic optimization techniques for MPI parallel applications

Data Locality Aware Strategy for Two-Phase Collective I/O

Contact Info

Product

Resources

About