Parallel performance prediction using lost cycles analysis

Abstract

Most performance debugging and tuning of parallel programs is based on the \measure-modify" approach, which is heavily dependent on detailed m e asurements of programs during execution. This approach is extremely time-consuming and does not lend itself to predicting performance under varying conditions. Analytic modeling and scalability analysis provide predictive power, but are not widely used i n p r actice, due primarily to their emphasis on asymptotic behavior and the di culty of developing accurate models that work for real-world programs. In this paper we describe a s e t o f t o ols for performance tuning of parallel programs that bridges this gap between measurement and modeling.Our approach is based o n lost cycles analysis, which involves measurement and modeling of all sources of overhead in a parallel program. We rst describe a t o ol for measuring overheads in parallel programs that we have incorporated i n t o t h e runtime environment for Fortran programs on the Kendall Square KSR1. We then describe a t o ol that ts these overhead measurements to analytic forms. We illustrate the use of these tools by analyzing the performance t r adeo s among parallel implementations of 2D FFT. These examples show how our tools enable programmers to develop accurate performance m o dels of parallel applications without requiring extensive performance m o deling expertise.

A high‐performance computing framework for Monte Carlo ocean color simulations

“…The study is part of a wide range of performance modeling solutions for optimizing time and space in simulation-driven investigations for Geosciences, as reviewed next. Performance modeling and prediction in the literature can be roughly divided into three classes: Fully analytical modeling based on in-depth knowledge of target computer programs and environments [41][42][43]; semi-analytical modeling based on a priori knowledge of the programs and environments to identify appropriate model expressions, and least-squares fitting of model coefficients to observed performance data [44][45][46][47][48][49][50]; and empirical performance modeling relying on very general model expressions whose coefficients are learned from data [48,51]. The key element exploited for MOX performance modeling is the identification of linear and nonlinear components in the photon tracing time.…”

Methodology for modelling SPMD hybrid parallel computation

“…Worley et al relate sensitivity of application performance to communication protocol [24,25]. Crovella et al measure and account for all sources of parallel overhead [26]. The authors model overhead by fitting trial functions to timing data.…”