Performance Comparison of MPI Implementations over InfiniBand, Myrinet and Quadrics

Liu, Jiuxing; Chandrasekaran, B.; Wu, Jiesheng; Jiang, Weihang; Kini, Sushmitha P.; Yu, Weikuan; Buntinas, Darius; Wyckoff, P.; Panda, Dhabaleswar K.

doi:10.1145/1048935.1050208

Cited by 95 publications

(80 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of RDMA over InfiniBand for MPI communication has been widely accepted as the preferred method for remote data copy operations due to several advantages previously studied [4]. The initiating task performs a copy of the remote buffer into the local buffer.…”

Section: Related Workmentioning

confidence: 99%

Profile Guided MPI Protocol Selection for Point-to-Point Communication Calls

Marathe

Lowenthal

et al. 2011

2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and PHD Forum

View full text Add to dashboard Cite

Abstract-Improving communication performance is critical to achieving high performance in message-passing programs. Designing new, efficient protocols to realize point-to-point and collective communication operations has therefore been an active area of research. However, the best protocol for a given communication routine is both application and architecture specific.This paper contributes a new method of selection of the optimal protocol for a given point-to-point communication pair. Our technique analyzes the MPI communication call profile of an application and uses a computation and communication model we have developed to choose the proper protocol for each communication phase. We have applied our system to MPI applications such as CG, Sweep3D and Sparse Matrix multiplication, as well as synthetic applications. Our scheme yields an improvement in total execution time of up to 20% compared to MVAPICH2 and up to 3.2% compared to the best, highly optimized communication protocol for the real applications. Furthermore, experiments on the synthetic applications show that the savings can be much more pronounced.

show abstract

Section: Related Workmentioning

confidence: 99%

Profile Guided MPI Protocol Selection for Point-to-Point Communication Calls

Marathe

Lowenthal

et al. 2011

2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and PHD Forum

View full text Add to dashboard Cite

show abstract

“…In order to evaluate the impact of threads on network latency, we use the multi-threaded latency test that is included in the OSU Micro Benchmark suite [14]. This benchmark performs ping-pong test with a single sender and multiple receiver threads.…”

Section: B Impact Of Threads On Latencymentioning

confidence: 99%

A scalable and generic task scheduling system for communication libraries

Denis

2009

2009 IEEE International Conference on Cluster Computing and Workshops

View full text Add to dashboard Cite

“…Related Work Previous work has been done on understanding the communication and non-communication overheads (in the context of MPI) on various architectures [13], [14], [15], [16], [17], [18]. However, none of this work looks at the network saturation behavior that is becoming increasingly important with system size, which is the focus of this paper.…”

Section: Nearest-neighbor Communicationmentioning

confidence: 99%

Understanding Network Saturation Behavior on Large-Scale Blue Gene/P Systems

Balaji

Naik

Desai

2009

2009 15th International Conference on Parallel and Distributed Systems

View full text Add to dashboard Cite

Abstract-As researchers continue to architect massivescale systems, it is becoming clear that these systems will utilize a significant amount of shared hardware between processing units. Systems such as the IBM Blue Gene (BG) and Cray XT have started utilizing flat (i.e., scalable) networks, which differ from switched fabrics in that they use a 3D torus or similar topology. This allows the network to grow only linearly with system scale, instead of the super linear growth needed for full fat-tree switched topologies, but at the cost of increased network sharing between processing nodes. While in many cases a full fat-tree is an over estimate of the needed bisectional bandwidth, it is not clear whether the other extreme of a flat topology is sufficient to move data around the network efficiently. In this paper, we study the network behavior of the IBM BG/P using several application communication kernels, and we monitor network congestion behavior based on detailed hardware counters. Our studies scale from small systems to 8 racks (32,768 cores) of BG/P and provide insights into the network communication characteristics of the system.As researchers continue to architect massivescale systems, it is becoming clear that these systems will utilize a significant amount of shared hardware between processing units. Systems such as the IBM Blue Gene (BG) and Cray XT have started utilizing flat (i.e., scalable) networks, which differ from switched fabrics in that they use a 3D torus or similar topology. This allows the network to grow only linearly with system scale, instead of the super linear growth needed for full fat-tree switched topologies, but at the cost of increased network sharing between processing nodes. While in many cases a full fat-tree is an over estimate of the needed bisectional bandwidth, it is not clear whether the other extreme of a flat topology is sufficient to move data around the network efficiently. In this paper, we study the network behavior of the IBM BG/P using several application communication kernels, and we monitor network congestion behavior based on detailed hardware counters. Our studies scale from small systems to 8 racks (32,768 cores) of BG/P and provide insights into the network communication characteristics of the system.

show abstract

Performance Comparison of MPI Implementations over InfiniBand, Myrinet and Quadrics

Cited by 95 publications

References 21 publications

Profile Guided MPI Protocol Selection for Point-to-Point Communication Calls

Profile Guided MPI Protocol Selection for Point-to-Point Communication Calls

A scalable and generic task scheduling system for communication libraries

Understanding Network Saturation Behavior on Large-Scale Blue Gene/P Systems

Contact Info

Product

Resources

About