Toward Performance Models of MPI Implementations for Understanding Application Scaling Issues

Abstract

Abstract. Designing and tuning parallel applications with MPI, particularly at large scale, requires understanding the performance implications of different choices of algorithms and implementation options. Which algorithm is better depends in part on the performance of the different possible communication approaches, which in turn can depend on both the system hardware and the MPI implementation. In the absence of detailed performance models for different MPI implementations, application developers often must select methods and tune codes without the means to realistically estimate the achievable performance and rationally defend their choices. In this paper, we advocate the construction of more useful performance models that take into account limitations on network-injection rates and effective bisection bandwidth. Since collective communication plays a crucial role in enabling scalability, we also provide analytical models for scalability of collective communication algorithms, such as broadcast, allreduce, and all-to-all. We apply these models to an IBM Blue Gene/P system and compare the analytical performance estimates with experimentally measured values. MotivationPerformance modeling of parallel applications leads to an understanding of their running time on parallel systems. To develop a model for an existing application or algorithm, one typically constructs a dependency graph of computations and communications from the start of the algorithm (input) to the end of the algorithm (output). This application model can then be matched to a machine model in order to estimate the run time of the algorithm on a particular architecture.Performance models can be used to make important early decisions about algorithmic choices. For example, to compute a three-dimensional Fast Fourier Transformation (3d FFT), one can either use a one-dimensional decomposition where each process computes full planes (2d FFTs) or a two-dimensional decomposition where each process computes sets of pencils (1d FFTs). If we assume

Performance modeling for systematic performance tuning

“…Collective communications are typically simple to capture (only needs the type of collective operation [26]). However, it is important to also record information about the communicator size (it might be as simple as the total number of processes).…”

Final Report for Enhancing the MPI Programming Model for PetaScale Systems

“…The use of performance modeling to better understand scaling issues has become an important part of the project; some issues, including ones relating to MPI, were discussed in [18], which was presented at SC'11. Previous results, for I/O [16] and for performance correctness [31,19], have appeared, and a "Best Paper" at EuroMPI'10 on load balancing in hybrid MPI--OpenMP programs in [20].…”

Bridging Performance Analysis Tools and Analytic Performance Modeling for HPC

“…This approach relies on memory profiles of the application and automated, simulation-based prediction. Hoefler et al define strategies to trade the accuracy and complexity for modeling the performance of Message Passing Interface implementations [7].…”