Reproducible MPI Benchmarking is Still Not as Easy as You Think

Hunold, Sascha; Carpen-Amarie, Alexandra

doi:10.1109/tpds.2016.2539167

Cited by 35 publications

(12 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The executions are interleaved among the pseudo‐applications. This number of executions was established as indicated in the work of Hunold and Carpen‐Amarie . In this study, the authors perform experiments that show that this is the minimum number of executions of MPI in order to obtain statistically acceptable results.…”

Section: Resultsmentioning

confidence: 99%

PAMPAR: A new parallel benchmark for performance and energy consumption evaluation

Garcia

Schepke

Girardi

2019

Concurrency and Computation

View full text Add to dashboard Cite

Summary This paper presents PAMPAR, a new benchmark to evaluate the performance and energy consumption of different Parallel Programming Interfaces (PPIs). The benchmark is composed of 11 algorithms implemented in PThreads, OpenMP, MPI‐1, and MPI‐2 (spawn) PPIs. Previous studies have used some of these pseudo‐applications to perform this type of evaluation in different architectures since there is no benchmark that offers this variety of PPIs and communication models. In this work, we measure the energy and performance of each pseudo‐application in a single architecture, varying the number of threads/processes. We also organize the pseudo‐applications according to their memory accesses, floating‐point operations, and branches. The goal is to show that this set of pseudo‐applications has enough features to build a parallel benchmark. The results show that there is no single best case that provides both better performance and low energy consumption in the presented scenarios. Moreover, the pseudo‐applications usage of the system resources are different enough to represent different scenarios and be efficient as a benchmark.

show abstract

Section: Resultsmentioning

confidence: 99%

PAMPAR: A new parallel benchmark for performance and energy consumption evaluation

Garcia

Schepke

Girardi

2019

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…Another consideration here, however, is that simulated executions are "clean". In other terms, they are reproducible and well-structured because they do not suffer from idiosyncratic effects seen in real-world platforms [11]- [14]. Therefore, it is possible to observe clearly performance effects of variations in algorithms, implementations, and platform configurations.…”

Section: B Sample Assignments and Usefulness Of Simulationmentioning

confidence: 96%

“…Third, real-world platforms are known to be "noisy" [11]- [14]. The observed performance of a program is subject to many effects, some deterministic and some non-deterministic, which make performance difficult to understand without deep knowledge about hardware/software infrastructures and without solid experimental and analytical skills.…”

Section: Introductionmentioning

confidence: 99%

SMPI Courseware: Teaching Distributed-Memory Computing with MPI in Simulation

Casanova¹,

Legrand²,

Quinson³

et al. 2018

2018 IEEE/ACM Workshop on Education for High-Performance Computing (EduHPC)

View full text Add to dashboard Cite

It is typical in High Performance Computing (HPC) courses to give students access to HPC platforms so that they can benefit from hands-on learning opportunities. Using such platforms, however, comes with logistical and pedagogical challenges. For instance, a logistical challenge is that access to representative platforms must be granted to students, which can be difficult for some institutions or course modalities; and a pedagogical challenge is that hands-on learning opportunities are constrained by the configurations of these platforms.A way to address these challenges is to instead simulate program executions on arbitrary HPC platform configurations. In this work we focus on simulation in the specific context of distributed-memory computing and MPI programming education. While using simulation in this context has been explored in previous works, our approach offers two crucial advantages. First, students write standard MPI programs and can both debug and analyze the performance of their programs in simulation mode. Second, large-scale executions can be simulated in short amounts of time on a single standard laptop computer. This is possible thanks to SMPI, an MPI simulator provided as part of SimGrid. After detailing the challenges involved when using HPC platforms for HPC education and providing background information about SMPI, we present SMPI Courseware. SMPI Courseware is a set of in-simulation assignments that can be incorporated into HPC courses to provide students with hands-on experience for distributed-memory computing and MPI programming learning objectives. We describe some these assignments, highlighting how simulation with SMPI enhances the student learning experience.

show abstract

“…For the analysis shown in the present paper, we have used our own benchmark suite called ReproMPI 2 , which allows to record raw data (the latency of every single measurement) from each experiment [5]. In contrast to other benchmark suites, it refrains l [obs] = G _T () -t from performing any kind of data aggregation (e.g., computation of means) or data removal (e.g., discarding the rst X measurements for "warming up" the system).…”

Section: Tuning Work Owmentioning

confidence: 99%

“…MPI timing procedure ([5]). 1: procedure T _MPI_ (func, msize, nrep) // func -MPI function; msize -message size; nrep -nb.…”

mentioning

confidence: 99%

Autotuning MPI Collectives using Performance Guidelines

Hunold

Carpen-Amarie

2018

Proceedings of the International Conference on High Performance Computing in Asia-Pacific Region

Self Cite

View full text Add to dashboard Cite

MPI collective operations provide a standardized interface for performing data movements within a group of processes. The e ciency of collective communication operations depends on the actual algorithm, its implementation, and the speci c communication problem (type of communication, message size, number of processes). Many MPI libraries provide numerous algorithms for speci c collective operations. The strategy for selecting an e cient algorithm is often times prede ned (hard-coded) in MPI libraries, but some of them, such as Open MPI, allow users to change the algorithm manually. Finding the best algorithm for each case is a hard problem, and several approaches to tune these algorithmic parameters have been proposed. We use an orthogonal approach to the parameter-tuning of MPI collectives, that is, instead of testing individual algorithmic choices provided by an MPI library, we compare the latency of a speci c MPI collective operation to the latency of semantically equivalent functions, which we call the mock-up implementations. The structure of the mock-up implementations is de ned by selfconsistent performance guidelines. The advantage of this approach is that tuning using mock-up implementations is always possible, whether or not an MPI library allows users to select a speci c algorithm at run-time. We implement this concept in a library called PGMPITuneLib, which is layered between the user code and the actual MPI implementation. This library selects the best-performing algorithmic pattern of an MPI collective by intercepting MPI calls and redirecting them to our mock-up implementations. Experimental results show that PGMPITuneLib can signi cantly reduce the latency of MPI collectives, and also equally important, that it can help identifying the tuning potential of MPI libraries.

show abstract

Reproducible MPI Benchmarking is Still Not as Easy as You Think

Cited by 35 publications

References 18 publications

PAMPAR: A new parallel benchmark for performance and energy consumption evaluation

PAMPAR: A new parallel benchmark for performance and energy consumption evaluation

SMPI Courseware: Teaching Distributed-Memory Computing with MPI in Simulation

Autotuning MPI Collectives using Performance Guidelines

Contact Info

Product

Resources

About