Exploring Techniques for the Analysis of Spontaneous Asynchronicity in MPI-Parallel Applications

Abstract

This paper studies the utility of using data analytics and machine learning techniques for identifying, classifying, and characterizing the dynamics of large-scale parallel (MPI) programs. To this end, we run microbenchmarks and realistic proxy applications with the regular compute-communicate structure on two different supercomputing platforms and choose the per-process performance and MPI time per time step as relevant observables. Using principal component analysis, clustering techniques, correlation functions, and a new "phase space plot," we show how desynchronization patterns (or lack thereof) can be readily identified from a data set that is much smaller than a full MPI trace. Our methods also lead the way towards a more general classification of parallel program dynamics.

Making Applications Faster by Asynchronous Execution: Slowing Down Processes or Relaxing MPI Collectives

“…strongly saturating because of its very low computational intensity. In [7]) we also employed PISOLVER, which numerically evaluates 1 0 4/(1 + x 2 ) dx using the midpoint rule. This is a purely compute-bound workload dominated by floating-point divides and scales perfectly across cores.…”

“…For the MST and LBM cases we concentrate on timelines and phase-space plots, which can be regarded as more explorative data analysis techniques. The phase-space plot was introduced by us in [7]. Furthermore, two metrics are examined: performance per process and MPI time per process.…”

“…Research contributions. The focus of our previous publication [7] was on the exploration of various data analysis techniques; here we concentrate on the analysis of applications, particularly employing the new technique of phase-space analysis for temporal evolution. The impact of MPI asynchronous execution on performance is explored, particularly to distinguish between parallel codes that are compute bound and those that are memory bandwidth limited.…”

“…[22,9,18,27]. [7] Research techniques and metrics. This paper is a follow-up of [7], where we investigated techniques and metrics for quantifying asynchronicity by observing the behavior of MPI waiting times.…”

“…[7] Research techniques and metrics. This paper is a follow-up of [7], where we investigated techniques and metrics for quantifying asynchronicity by observing the behavior of MPI waiting times. Five data analytics techniques (timelines, histograms, compact timelines, correlation coefficients [26], and phase-space plots) and two machine learning techniques (Principal Component analysis [16] and K-means clustering 1 ) were covered.…”