Profile Guided MPI Protocol Selection for Point-to-Point Communication Calls

Abstract

Abstract-Improving communication performance is critical to achieving high performance in message-passing programs. Designing new, efficient protocols to realize point-to-point and collective communication operations has therefore been an active area of research. However, the best protocol for a given communication routine is both application and architecture specific.This paper contributes a new method of selection of the optimal protocol for a given point-to-point communication pair. Our technique analyzes the MPI communication call profile of an application and uses a computation and communication model we have developed to choose the proper protocol for each communication phase. We have applied our system to MPI applications such as CG, Sweep3D and Sparse Matrix multiplication, as well as synthetic applications. Our scheme yields an improvement in total execution time of up to 20% compared to MVAPICH2 and up to 3.2% compared to the best, highly optimized communication protocol for the real applications. Furthermore, experiments on the synthetic applications show that the savings can be much more pronounced.

An Evaluation of One-Sided and Two-Sided Communication Paradigms on Relaxed-Ordering Interconnect

“…The registration decision is more challenging at the receiver side because of message size ambiguity. Researchers have tried to optimize [11] message matching and provide early registration at the receiver when possible.…”

“…Woodall et al [14] also describe RDMA optimizations for MPI such as registration caches and pipelining of registration with the data transmission for large messages. Optimizing memory registration is also discussed by Marathe et al [11] which propose an early registration strategy with an associated performance model. Based on our measurements, Cray MPI does not use this approach.…”