Multi‐installment divisible load processing in heterogeneous distributed systems

Abstract

SUMMARYDivisible loads are parallel applications with fine granularity and negligible data dependencies. Such computations can be divided into parts of arbitrary sizes and processed independently in parallel. The load distribution process incurs considerable communication delays. To reduce processor waiting time during the computation initialization phase, the load is distributed in multiple small installments rather than in one big chunk. In this paper we analyze multi-installment divisible load processing in heterogeneous distributed systems. Scheduling divisible loads in heterogeneous systems is hard because the sizes of the installments should be adjusted to the communication and computation capabilities of the system. We show that ignoring heterogeneity of the distributed system may result in arbitrarily bad solutions. Two algorithms are proposed to gear the load chunk sizes to different communication and computation speeds: an optimization branch-and-bound algorithm and a heuristic based on a genetic search method. The running times of both methods and the quality of the solutions are compared. Then, we use these algorithms to study the features of the multi-installment divisible load scheduling problem. We demonstrate that it has both combinatorial and algebraic nature, and that optimum solutions are harder to find with the growing heterogeneity of the system.

New method for scheduling heterogeneous multi-installment systems

“…Many articles were presented in this field [8][9][10]. Complexity of job scheduling in a multi-installment system is very high.…”

Adaptive structured parallelism for distributed heterogeneous architectures: a methodological approach with pipelines and farms

“…Traditional strategies for task scheduling in distributed systems [17,[27][28][29][30] rely on system simulators, dedicated configurations, and/or performance estimators to model the general system, particularly to characterize the background load in terms of its job arrival rate. While much can be said about the reproducibility of their results, one may argue that they artificially create tractable evaluation scenarios for their scheduling policies.…”

Adaptive statistical scheduling of divisible workloads in heterogeneous systems

“…The Uniform MultiRound algorithm assumes that every node receives decreasing, fixed task sizes in every round and provides an approximation to the optimal number of rounds by minimising the application makespan in a simulated environment ). Drozdowski and Lawenda (2007) tackle the problem as an optimisation of the application makespan but relax the assumption on fixed task sizes, approximating the solution via branch-and-bound and genetic algorithms on a simulated heterogeneous environment.…”