High Performance RDMA-Based MPI Implementation over InfiniBand

Liu, Jiuxing; Wu, Jiesheng; Panda, Dhabaleswar K.

doi:10.1023/b:ijpp.0000029272.69895.c1

Cited by 204 publications

(133 citation statements)

References 0 publications

Supporting

Mentioning

133

Contrasting

Order By: Relevance

“…It is also similar to implementations that have been built on networks that have remote DMA (RDMA) capability, such as those described in [4,10,17]. However, the traveling thread model is able to support some features of MPI much more efficiently.…”

Section: Related Workmentioning

confidence: 97%

Implications of a PIM architectural model for MPI

Rodrigues

Murphy

Kogge

et al. 2003

Proceedings IEEE International Conference on Cluster Computing CLUSTR-03

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 97%

Implications of a PIM architectural model for MPI

Rodrigues

Murphy

Kogge

et al. 2003

Proceedings IEEE International Conference on Cluster Computing CLUSTR-03

View full text Add to dashboard Cite

show abstract

“…The data are stored into the receive queue in the HCA of the receiver node. The HCA writes the received data into the receiver process' memory (step iii) [27]- [29].…”

Section: Inter-node Communicationmentioning

confidence: 99%

An Efficient I/O Aggregator Assignment Scheme for Multi-Core Cluster Systems

Cha

2013

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYAs the number of nodes in high-performance computing (HPC) systems increases, parallel I/O becomes an important issue: collective I/O is the specialized parallel I/O that provides the function of singlefile based parallel I/O. Collective I/O in most message passing interface (MPI) libraries follows a two-phase I/O scheme in which the particular processes, namely I/O aggregators, perform important roles by engaging the communications and I/O operations. This approach, however, is based on a single-core architecture. Because modern HPC systems use multi-core computational nodes, the roles of I/O aggregators need to be re-evaluated. Although there have been many previous studies that have focused on the improvement of the performance of collective I/O, it is difficult to locate a study regarding the assignment scheme for I/O aggregators that considers multi-core architectures. In this research, it was discovered that the communication costs in collective I/O differed according to the placement of the I/O aggregators, where each node had multiple I/O aggregators. The performance with the two processor affinity rules was measured and the results demonstrated that the distributed affinity rule used to locate the I/O aggregators in different sockets was appropriate for collective I/O. Because there may be some applications that cannot use the distributed affinity rule, the collective I/O scheme was modified in order to guarantee the appropriate placement of the I/O aggregators for the accumulated affinity rule. The performance of the proposed scheme was examined using two Linux cluster systems, and the results demonstrated that the performance improvements were more clearly evident when the computational node of a given cluster system had a complicated architecture. Under the accumulated affinity rule, the performance improvements between the proposed scheme and the original MPI-IO were up to approximately 26.25% for the read operation and up to approximately 31.27% for the write operation. key words: collective I/O, parallel I/O, processor affinity IntroductionAs the size of a problem increases, many scientific applications generate a large number of file-I/O operations. Today's parallel programming paradigms provide some I/O methods for scientific applications, and previous studies [2]-[4] demonstrated the importance of single-file based parallel I/O, especially collective I/O.Collective I/O in message-passing interface (MPI) follows the two-phase I/O scheme that consists of an I/O phase and a data exchange phase [5]. In the two-phase I/O, the specialized process called I/O aggregator is engaged in the both phases. In other words, because the role of I/O aggregator is to collect or distribute I/O data to other clients, collective I/O performance can be affected by the ability of the I/O aggregator. In this study, we describe the effect of processor affinity in collective I/O considering multi-core cluster systems. Especially, we explain the relationship between the placement of I/O aggregators in each node a...

show abstract

“…MPI communication over RDMA has been successfully implemented [5]. The use of RDMA over InfiniBand for MPI communication has been widely accepted as the preferred method for remote data copy operations due to several advantages previously studied [4].…”

Section: Related Workmentioning

confidence: 99%

Profile Guided MPI Protocol Selection for Point-to-Point Communication Calls

Marathe

Lowenthal

et al. 2011

2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and PHD Forum

View full text Add to dashboard Cite

Abstract-Improving communication performance is critical to achieving high performance in message-passing programs. Designing new, efficient protocols to realize point-to-point and collective communication operations has therefore been an active area of research. However, the best protocol for a given communication routine is both application and architecture specific.This paper contributes a new method of selection of the optimal protocol for a given point-to-point communication pair. Our technique analyzes the MPI communication call profile of an application and uses a computation and communication model we have developed to choose the proper protocol for each communication phase. We have applied our system to MPI applications such as CG, Sweep3D and Sparse Matrix multiplication, as well as synthetic applications. Our scheme yields an improvement in total execution time of up to 20% compared to MVAPICH2 and up to 3.2% compared to the best, highly optimized communication protocol for the real applications. Furthermore, experiments on the synthetic applications show that the savings can be much more pronounced.

show abstract

High Performance RDMA-Based MPI Implementation over InfiniBand

Cited by 204 publications

References 0 publications

Implications of a PIM architectural model for MPI

Implications of a PIM architectural model for MPI

An Efficient I/O Aggregator Assignment Scheme for Multi-Core Cluster Systems

Profile Guided MPI Protocol Selection for Point-to-Point Communication Calls

Contact Info

Product

Resources

About