Scheduling divisible loads on star and tree networks: results and open problems

Abstract

Many applications in scientific and engineering domains are structured as large numbers of independent tasks with low granularity. These applications are thus amenable to straightforward parallelization, typically in master-worker fashion, provided that efficient scheduling strategies are available. Such applications have been called divisible loads because a scheduler may divide the computation among worker processes arbitrarily, both in terms of number of tasks and of task sizes. Divisible load scheduling has been an active area of research for the last twenty years. A vast literature offers results and scheduling algorithms for various models of the underlying distributed computing platform. Broad surveys are available that report on accomplishments in the field. By contrast, in this paper we propose a unified theoretical perspective that synthesizes previously published results, several novel results, and open questions, in a view to foster novel divisible load scheduling research. Specifically, we discuss both one-round and multi-round algorithms, and we restrict our scope to the popular star and tree network topologies, which we study with both linear and affine cost models for communication and computation.Keywords: parallel computing, scheduling, divisible load Résumé De nombreuses applications scientifiques se découpent naturellement en un grand nombre de tâches indépendantes avec une faible granularité. Ces applications se parallélisent naturellementà l'aide d'une approche maître/esclave. De telles applications relèvent du modèle des tâches divisibles car un ordonnanceur peut diviser les calculs sur les différents processeurs disponibles,à la fois en terme de nombre de tâches maiś egalement en terme de taille des tâches. L'ordonnancement de tâches divisibles aété un domaine de recherche actif durant les vingts dernières années. On trouve donc dans la littérature de nombreux résultats et algorithmes d'ordonnancement pour différents modèles de plates-formes. A la différence desétats de l'art déja existant sur le sujet, ce rapport propose une nouvelle approche permettant d'unifier et de retrouver les résultats de la littérature, de proposer de nouveaux résultats et d'ouvrir de nouveaux problèmes. Plus précisément, nous présentons les distributions en une seule tournée et en plusieurs tournées et nous restreignons aux topologies populaires enétoile et en arborescence, que nous nouś etudionsà l'aide de coût de calculs et de communications linéaires puis affines.

Scheduling Divisible Loads with Return Messages on Heterogeneous Master-Worker Platforms

“…Extending this result to a star network (with different master-worker link bandwidths), but still (1) without return messages and (2) with a linear cost model, has been achieved only recently [5]. The proof basically goes along the same steps as for a bus network, but the main additional difficulty was to find the optimal ordering of the messages from the master to the workers.…”

PPDD: scheduling multi-site divisible loads in single-level tree networks

“…This study used bus networks as the underlying topology. Beaumont et al consolidates the results for single-level tree and bus topologies and presents extensive discussions on some open problems in this domain [25]. A few new applications and solutions in DLT have been investigated in recent years, e.g., bioinformatics [26], multimedia streaming [27], sensor networks [28,29], economic and game-theoretic approaches [30,31].…”

Setting up Clusters of Computing Units to Process Several Data Streams Efficiently

“…According to the master-workers paradigm, the processing units which distribute the load act as masters and the computation units which do the processing act as workers [6]. Besides, we assume that the computation units have an unlimited buffering capability, unlike [7].…”