Quantifying the effectiveness of load balance algorithms

Abstract

Load balance is critical for performance in large parallel applications. An imbalance on today's fastest supercomputers can force hundreds of thousands of cores to idle, and on future exascale machines this cost will increase by over a factor of a thousand. Improving load balance requires a detailed understanding of the amount of computational load per process and an application's simulated domain, but no existing metrics sufficiently account for both factors. Current load balance mechanisms are often integrated into applications and make implicit assumptions about the load. Some strategies place the burden of providing accurate load information, including the decision on when to balance, on the application. Existing application-independent mechanisms simply measure the application load without any knowledge of application elements, which limits them to identifying imbalance without correcting it.Our novel load model couples abstract application information with scalable measurements to derive accurate and actionable load metrics. Using these metrics, we develop a cost model for correcting load imbalance. Our model enables comparisons of the effectiveness of load balancing algorithms in any specific imbalance scenario. Our model correctly selects the algorithm that achieves the lowest runtime in up to 96% of the cases, and can achieve a 19% gain over selecting a single balancing algorithm for all cases.

Automated Load Balancing Invocation Based on Application Characteristics

“…[11] in press. The primary focus of this work is on selection of load balancing strategy based on simulation of multiple strategies.…”

“…Based on the application characteristics observed at runtime and a set of guiding principles, MetaBalancer relieves the application programmer from the critical task of deciding when the load balancer should be invoked. Unlike many existing models, which rely only on the most recent data and do not make predictions based on dynamic nature of applications [10,11], Meta-Balancer continuously monitors the application and predicts load behavior. Using a linear prediction model on the collected information, Meta-Balancer predicts the time steps (or iterations) at which load balancing should be performed for optimal performance.…”

A simulated annealing algorithm for zoning in planning using parallel computing

“…In order to balance the computational load, the clusters were distributed among the different threads so that the number of plots in each thread was as balanced as possible. Based on the definitions of load metrics in parallel computers (Pearce, Gamblin, de Supinski, Schulz, & Amato, 2012), for a given distribution of clusters, the balance can be defined as,…”

Resisting Skew-Accumulation for Time-Stepped Applications in the Cloud via Exploiting Parallelism

“…Pearce et al [26] proposed a load model for load balance algorithms based on application elements and their interactions to guide the selection of load balance algorithms.…”