Dynamic Load Balancing on Dedicated Heterogeneous Systems

Abstract

Abstract. Parallel computing in heterogeneous environments is drawing considerable attention due to the growing number of these kind of systems. Adapting existing code and libraries to such systems is a fundamental problem. The performance of this code is affected by the large interdependence between the code and these parallel architectures. We have developed a dynamic load balancing library that allows parallel code to be adapted to heterogeneous systems for a wide variety of problems. The overhead introduced by our system is minimal and the cost to the programmer negligible. The strategy was validated on several problems to confirm the soundness of our proposal.

FLEX-MPI: An MPI Extension for Supporting Dynamic Load Balancing on Heterogeneous Non-dedicated Systems

“…Galindo et al [6] present a model based on the relative computing power, a metric which is obtained by measuring the execution time invested by a processor in performing a given computation. The computation is measured as the number of rows of a dense matrix, an inaccurate model when it comes to sparse data structures.…”

“…Their approach only considers executions on dedicated systems. ALBIC [7] is a system based on [6] which measures the system load by collecting performance metrics at runtime. However, this technique is intrusive since to collect this data and feed it to the monitoring system they add a specific module in the Linux kernel.…”

Dynamic Load Balancing of Parallel Computational Iterative Routines on Platforms with Memory Heterogeneity

“…These strategies are restricted to applications with pre-determined workload and cannot be applied to such iterative routines as adaptive mesh refinement [7], for which the amount of computation data grows unpredictably. Dynamic algorithms [8,9,10,11,12] do not require a priori information and can be used with a wider class of parallel applications. In addition, dynamic algorithms can be deployed on non-dedicated platforms.…”

“…In non-centralised algorithms [11,12], load is migrated locally between neighbouring processors, while in centralised ones [4,5,6,8,9,10], load is distributed based on global load information. Non-centralized algorithms are slower to converge.…”

“…They target parallel routines consisting of independent tasks and schedule them on sharedmemory platforms. Predicting-the-future algorithms [4,5,6,8] can distribute both tasks and data by predicting future performance based on past information. They are suitable for data-intensive iterative routines and any parallel computational platform.…”