A Batch Task Migration Approach for Decentralized Global Rescheduling

Abstract

Effectively mapping tasks of High Performance Computing (HPC) applications on parallel systems is crucial to assure substantial performance gains. As platforms and applications grow, load imbalance becomes a priority issue. Even though centralized rescheduling has been a viable solution to mitigate this problem, its efficiency is not able to keep up with the increasing size of shared memory platforms. To efficiently solve load imbalance today, and in the years to come, we should prioritize decentralized strategies developed for large scale platforms. In this paper, we propose our Batch Task Migration approach to improve decentralized global rescheduling, ultimately reducing communication costs and preserving task locality. We implemented and evaluated our approach in two different parallel platforms, using both synthetic workloads and a molecular dynamics (MD) benchmark. Our solution was able to achieve speedups of up to 3.75 and 1.15 on rescheduling time, when compared to other centralized and distributed approaches, respectively. Moreover, it improved the execution time of MD by factors up to 1.34 and 1.22 when compared to a scenario without load balancing on two different platforms.

Distributed Memory Graph Representation for Load Balancing Data: Accelerating Data Structure Generation for Decentralized Scheduling

“…The increasing performance demand of parallel applications in HPC environments creates a need for fast, reliable and efficient schedulers. Both our experiments and the literature [10,12,13,22] indicate that parallel and distributed load balancers are the best candidate to fulfill this role, especially when dynamic rescheduling is required. We believe that DGM will bring more benefits to communication-aware strategies in the future, helping to achieve Exascale performance in distributed memory environments.…”

“…Traditionally, these algorithms used a diffusive approach [11], which consists of iteratively sending work to neighbors that carry a lighter workload. This leads to refinement-based approaches, which attempt to send work units to underloaded resources using probabilistic distributions (e.g., Grapevine [12]) or accumulating work to mitigate communication increases after migration (e.g., PackDrop [13]).…”

“…The use of graph partitioning to create homogeneous clusters of communicating tasks inside DGM may be used to migrate these aggregates of tasks, which should preserve the most affinity among them [26]. This approach would enhance the naive Batch Task Migration [13], which could lead to even more efficient communication-aware scheduling.…”

Adaptive Load Balancing based on Machine Learning for Iterative Parallel Applications

“…We may divide LB algorithms in two main categories: Global and Diffusive [10]. While global algorithms centralize system information to make scheduling decisions (most list scheduling approaches follow this methodology [11], [12]); diffusive algorithms will use local machine information to dissipate load among its neighbors or other machines in the system [13], [14], [15].…”

PackStealLB: A scalable distributed load balancer based on work stealing and workload discretization

“…Our approach reduces scheduling overhead by applying the following design: (i) execute scheduling steps in a distributed fashion; (ii) perform workload discretization to simplify decisions; (iii) group local tasks to avoid numerous fine-grained migrations and (iv) minimize the messages between scheduling actors. The discretization technique, called packing, extends a previous technique [6] with notions related to -Nash Equilibrium. We also present PackStealLB , a new distributed load balancing algorithm that uses packing and inherits ideas related to constrained [7,8] and randomized [9] Work Stealing (WS) heuristics.…”