Parallel Scheduling of DAGs under Memory Constraints

Abstract

Scientific workflows are frequently modeled as Directed Acyclic Graphs (DAG) of tasks, which represent computational modules and their dependencies in the form of data produced by a task and used by another one. This formulation allows the use of runtime systems which dynamically allocate tasks onto the resources of increasingly complex computing platforms. However, for some workflows, such a dynamic schedule may run out of memory by exposing too much parallelism. This paper focuses on the problem of transforming such a DAG to prevent memory shortage, and concentrates on shared memory platforms. We first propose a simple model of DAGs which is expressive enough to emulate complex memory behaviors. We then exhibit a polynomial-time algorithm that computes the maximum peak memory of a DAG, that is, the maximum memory needed by any parallel schedule. We consider the problem of reducing this maximum peak memory to make it smaller than a given bound by adding new fictitious edges, while trying to minimize the critical path of the graph. After proving this problem NP-complete, we provide an ILP solution as well as several heuristic strategies that are thoroughly compared by simulation on synthetic DAGs modeling actual computational workflows. We show that on most instances we are able to decrease the maximum peak memory at the cost of a small increase in the critical path, thus with little impact on quality of the final parallel schedule. Key-words: Ordonnancement parallèle de DAGs sous contraintes mémoireRésumé : Les applications de calcul scientifique sont souvent modélisées par des graphes de tâches orientés acycliques (DAG), qui représentent les tâches de calcul et leurs dépendances, sous la forme de données produites par une tâche et utilisées par une autre. Cette formulation permet l'utilisation d'API qui allouent dynamiquement les tâches sur les ressources de plateformes de calcul hétérogènes de plus en plus complexes. Cependant, pour certaines applications, un tel ordonnancement dynamique peut manquer de mémoire en exploitant trop de parallélisme. Cet article porte sur le problème consistant à transformer un tel DAG pour empêcher toute pénurie de mémoire, en se concentrant sur les plateformes à mémoire partagée. On propose tout d'abord un modèle simple de graphe qui est assez expressif pour émuler des comportements mémoires complexes. On expose ensuite un algorithme polynomial qui calcule le pic mémoire maximum d'un DAG, qui représente la mémoire maximale requise par tout ordonnancement parallèle. On considère ensuite le problème consistant à réduire ce pic mémoire maximal pour qu'il devienne plus petit qu'une borne donnée en rajoutant des arêtes fictives, tout en essayant de minimiser le chemin critique du graphe. Après avoir prouvé ce problème NPcomplet, on fournit un programme linéaire en nombres entiers le résolvant, ainsi que plusieurs stratégies heuristiques qui sont minitieusement comparées-sur des graphes synthétiques modélisant des applications de calcul réelles. On montre que sur la plupar...

Cilkmem: Algorithms for Analyzing the Memory High-Water Mark of Fork-Join Parallel Programs

“…The relationship between PCOP and MHWM was previously made explicit by Marchal et al [24], who applied algorithms for PCOP in order to design polynomial-time algorithms for computing the memory high-water mark of parallel algorithms. Because none of the known algorithms for PCOP run in (even close) to linear time, however, the memory high-water mark algorithms of [24] are too inefficient to be used in practice.…”

“…The relationship between PCOP and MHWM was previously made explicit by Marchal et al [24], who applied algorithms for PCOP in order to design polynomial-time algorithms for computing the memory high-water mark of parallel algorithms. Because none of the known algorithms for PCOP run in (even close) to linear time, however, the memory high-water mark algorithms of [24] are too inefficient to be used in practice. Additionally, in order to apply PCOP algorithms to the computation of memory high-water marks, [24] were forced to make a number of simplifying assumptions about the parallel programs being analyzed, and their algorithms require that the parallel programs be in what they call the simple-data-flow model.…”

Revisiting dynamic DAG scheduling under memory constraints for shared-memory platforms

“…As explained in the introduction, we have previously proposed a way to restrict the potentially large memory needed for the traversal of a task graphs by adding fictitious edges [12,13]. Our method consists in first computing the worst achievable memory of any parallel traversal, using either a linear program or a min-flow algorithm.…”

“…There are few existing studies that take dynamic memory footprint into account when scheduling task graphs, as detailed below in Section 2. In our previous work [12,13], we have proposed an approach to ensure that any dynamic schedule never exceeds the available memory. In a nutshell, the idea is to introduce fictitious dependencies in the task graph to cope with memory constraints: these additional edges restrict the set of valid schedules and, in particular, forbid the concurrent execution of too many memory-intensive tasks.…”

“…However, this previous work [12,13] does not account for resource limitation: there are only p processors, hence no more than p tasks can be processed concurrently. In terms of memory usage, ignoring resource limitation translates into considering too many potential cuts, thereby requiring too many fictitious edges, which unduly constraints the dynamic schedules.…”