OMPIO: A Modular Software Architecture for MPI I/O

Chaarawi, Mohamad; Gabriel, Edgar; Keller, Rainer; Graham, Richard L.; Bosilca, George; Dongarra, Jack

doi:10.1007/978-3-642-24449-0_11

Cited by 14 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Venkatesan et al [11] proposed general NBC I/O operations that became the basis of the proposal for MPI 3.1 standard described in Section II-B. The researchers implemented NBC I/O operations in the Open MPI I/O library (OMPIO) [13] using the libNBC library [5]. Their implementation leverages the concept of collective operations schedule used in the libNBC library, whereas our implementation exploits the state machine and the extended generalized request to keep track of progress of NBC I/O operations.…”

Section: Related Workmentioning

confidence: 99%

Implementation and Evaluation of MPI Nonblocking Collective I/O

Seo

Latham

Zhang

et al. 2015

2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

View full text Add to dashboard Cite

The well-known gap between relative CPU speeds and storage bandwidth results in the need for new strategies for managing I/O demands. In large-scale MPI applications, collective I/O has long been an effective way to achieve higher I/O rates, but it poses two constraints. First, although overlapping collective I/O and computation represents the next logical step toward a faster time to solution, MPI's existing collective I/O API provides only limited support for doing so. Second, collective routines (both for I/O and communication) impose a synchronization cost in addition to a communication cost. The upcoming MPI 3.1 standard will provide a new set of nonblocking collective I/O operations to satisfy the need of applications. We present here initial work on the implementation of MPI nonblocking collective I/O operations in the MPICH MPI library. Our implementation begins with the extended two-phase algorithm used in ROMIO's collective I/O implementation. We then utilize a state machine and the extended generalized request interface to maintain the progress of nonblocking collective I/O operations. The evaluation results indicate that our implementation performs as well as blocking collective I/O in terms of I/O bandwidth and is capable of overlapping I/O and other operations. We believe that our implementation can help users try nonblocking collective I/O operations in their applications.

show abstract

Section: Related Workmentioning

confidence: 99%

Implementation and Evaluation of MPI Nonblocking Collective I/O

Seo

Latham

Zhang

et al. 2015

2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

View full text Add to dashboard Cite

show abstract

“…The framework for nonblocking collectives is implemented in the open-source library libNBC. We utilize and extend libNBC in conjunction with Open MPI's OMPIO framework [7] to handle nonblocking collective I/O operations. The central concept in libNBC's design is the collective operation schedule.…”

Section: A Framework For Nonblocking Collective I/o Operationsmentioning

confidence: 99%

Design and Evaluation of Nonblocking Collective I/O Operations

Venkatesan

Chaarawi

Gabriel

et al. 2011

Recent Advances in the Message Passing Interface

Self Cite

View full text Add to dashboard Cite

Abstract. Nonblocking operations have successfully been used to hide network latencies in large scale parallel applications. This paper presents the challenges associated with developing nonblocking collective I/O operations, in order to help hiding the costs of I/O operations. We also present an implementation based on the libNBC library, and evaluate the benefits of nonblocking collective I/O over a PVFS2 file system for a micro-benchmark and a parallel image processing application. Our results indicate the potential benefit of our approach, but also highlight the challenges to achieve appropriate overlap between I/O and compute operations.

show abstract