Self-Consistent MPI Performance Guidelines

Abstract

Message passing using the Message Passing Interface (MPI) is at present the most widely adopted framework for programming parallel applications for distributed-memory and clustered parallel systems.For reasons of (universal) implementability, the MPI standard does not state any specific performance guarantees, but users expect MPI implementations to deliver good and consistent performance in the sense of efficient utilization of the underlying parallel (communication) system. For performance portability reasons, users also naturally desire communication optimizations performed on one parallel platform with one MPI implementation to be preserved when switching to another MPI implementation on another platform.We address the problem of ensuring performance consistency and portability by formulating performance guidelines and conditions that are desirable for good MPI implementations to fulfill. Instead of prescribing a specific performance model (which may be realistic on some systems, under some MPI protocol and algorithm assumptions, etc.), we formulate these guidelines by relating the performance of various aspects of the semantically strongly interrelated MPI standard to each other. Common-sense expectations, for instance, suggest that no MPI function should perform worse than a combination of other MPI functions that implement the same functionality, that no specialized function should perform worse than a more general function that can implement the same functionality, that no function with weak semantic guarantees should perform worse than a similar function with stronger semantics, and so on. Such guidelines may enable implementers to provide higher quality MPI implementations, minimize performance surprises, and eliminate the need for users to make special, non-portable optimizations by hand.We introduce and semi-formalize the concept of self-consistent performance guidelines for MPI, and provide a (non-exhaustive) set of such guidelines in a form that could be automatically verified by benchmarks and experiment-management tools. We present experimental results that show cases where guidelines are not satisfied in common MPI implementations, thereby indicating room for improvement in today's MPI implementations.

Lightweight Parallel Foundations: a model-compliant communication layer

“…MPI provides both and is the standard for high-performance message passing. It refrains from performance guarantees to not impose too large a burden on developers [12]. Indeed, many good implementations exist and support many different systems.…”

“…Indeed, many good implementations exist and support many different systems. While some argue a usable degree of performance portability is possible without guarantees [12], others have reported that commonly expected asymptotic bounds did not hold in practice [2,7].…”

Autotuning MPI Collectives using Performance Guidelines

“…The user's expectation is that the composition of the latter two functions should not be faster than executing the specialized one. In a more formal de nition [11], a performance guideline is de ned between two functionalities A and B, which semantically implement the same operation. If functionality A is the more specialized of the two, we can state that MPI_A(n) MPI_B(n), which means that A should complete faster than B for a comparable communication volume n. The communication volume n should be understood as the amount of "actual" data items.…”

“…Currently, PGMPITuneLib contains implementations of the performance guidelines listed in Equations (GL1)-(GL22), some of which were introduced before [6,11].…”

Effect of Parallelization, Execution Time and Inter-process Communication on Sorting Techniques using Message Passing Interface

“…Fangfa Fu et al [2] proposed a flexible and efficient programming model for NoC-Based MPSoC, MMPI (Multiprocessor Message Passing Interface). Jesper Larsson Träff et al [3] introduced and semi-formalized the concept of self-consistent performance guidelines for MPI. They have formulated these guidelines by relating the performance of various aspects of the semantically strongly interrelated MPI standard to each other.…”