DAGuE: A generic distributed DAG engine for High Performance Computing

Bosilca, George; Bouteiller, Aurélien; Danalis, Anthony; Hérault, Thomas; Lemarinier, Pierre; Dongarra, Jack

doi:10.1016/j.parco.2011.10.003

Cited by 201 publications

(55 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…PaRSEC [13,15] provides a highly efficient platform for distributed task based parallel execution. An annotated user code is here first (automatically) translated into a parametrized DAG representation.…”

Section: Introductionmentioning

confidence: 99%

DuctTeip: An efficient programming model for distributed task-based parallel computing

2019

View full text Add to dashboard Cite

Current high-performance computer systems used for scientific computing typically combine shared memory computational nodes in a distributed memory environment. Extracting high performance from these complex systems requires tailored approaches. Task based parallel programming has been successful both in simplifying the programming and in exploiting the available hardware parallelism for shared memory systems. In this paper we focus on how to extend task parallel programming to distributed memory systems. We use a hierarchical decomposition of tasks and data in order to accommodate the different levels of hardware. We test the proposed programming model on two different applications, a Cholesky factorization, and a solver for the Shallow Water Equations. We also compare the performance of our implementation with that of other frameworks for distributed task parallel programming, and show that it is competitive. arXiv:1801.03578v1 [cs.DC] 10 Jan 2018 to the computational work performed by one node, the 1×9 process grid has the smallest variance between nodes, and therefore also the lowest maximum work size. The 9 × 1 process grid leads to smaller maximum work size than the 3 × 3 process grid if B is large enough, but suffers from significant load imbalance in the case B = 18. In all cases, the work becomes more evenly distributed if the number of level 1 tasks B is larger. The statistics for communication and computation point in different directions, but when comparing with actual run times, we have found that the communication size is the most informative measure. Having a large total communication size is likely to be detrimental to performance as the risk of tasks left waiting for remote data increases as well as the risk of congestion of messages. A square process grid is the factor that has the largest impact. Regarding the block sizes, having a large B improves the load balance, but increases the amount of communication as well as the number of messages (another indicator that is not shown in the graphics).

show abstract

Section: Introductionmentioning

confidence: 99%

DuctTeip: An efficient programming model for distributed task-based parallel computing

2019

View full text Add to dashboard Cite

show abstract

“…Figure 5A shows the distribution only for reused subgraphs for the Eager-Match detection strategy with and without inspection (EM+I and EM, respectively). Each column shows the percentage of reused subgraphs with 2 to 4 nodes ( [2,4]), 5 to 9 nodes ( [5,9]), 10 to 49 nodes ( [10,49]), 50…”

Section: Of 12mentioning

confidence: 99%

DF‐DTM: Dynamic Task Memoization and reuse in dataflow

Rouberte

Sena

Nery

et al. 2018

Concurrency and Computation

View full text Add to dashboard Cite

Summary Instruction Reuse is a technique adopted in Von Neumann architectures that improves performance by avoiding redundant execution of instructions when the result to be produced can be obtained by searching an input/output memoization table for such instruction. Trace reuse can be applied to traces of instructions in a similar fashion. However, those techniques are yet to be studied in the context of the Dataflow model, which has been gaining traction in the high performance computing community due to its inherent parallelism. Dataflow programs are represented by directed graphs where nodes are instructions or tasks and edges denote data dependencies between tasks. This work presents Dataflow Dynamic Task Memoization (DF‐DTM), a technique that allows the reuse of both nodes and subgraphs in dataflow, which are analogous to instructions and traces, respectively. The potential of DF‐DTM is evaluated by a series of experiments that analyze the behavior of redundant tasks in five relevant benchmarks, where up to 99.70% of the instantiated tasks could be reused. Moreover, this paper evaluates how reuse rates can be affected by limiting subgraph size, memoization table size, task granularity, and problem size, showing that DF‐DTM can yield good reuse rates in more realistic environments.

show abstract

“…Effectively, the compiler performs static data flow analysis to convert an affine input serial program into a Direct Acyclic Graph (DAG), with program functions (kernels) as its nodes, and data dependency edges between kernels as its edges. Then, the run-time is responsible for addressing all DAG scheduling challenges, including background MPI data transfers between distributed resources [10].…”

Section: Compiler and Run-time Synergymentioning

confidence: 99%

“…A better solution would be to rely on a run-time system that can dynamically adapt the execution to the current hardware. DAGuE [10], which deploys dynamic micro-task scheduling, has been shown [11] to deliver portable high performance, on heterogeneous hardware for a class of regular problems, such as those occurring in linear algebra.…”

Section: Introductionmentioning

confidence: 99%

From Serial Loops to Parallel Execution on Distributed Systems

Bosilca

Bouteiller

Danalis

et al. 2012

Euro-Par 2012 Parallel Processing

Self Cite

View full text Add to dashboard Cite

Abstract. Programmability and performance portability are two major challenges in today's dynamic environment. Algorithm designers targeting efficient algorithms should focus on designing high-level algorithms exhibiting maximum parallelism, while relying on compilers and runtime systems to discover and exploit this parallelism, delivering sustainable performance on a variety of hardware. The compiler tool presented in this paper can analyze the data flow of serial codes with imperfectly nested, affine loop-nests and if statements, commonly found in scientific applications. This tool operates as the front-end compiler for the DAGuE run-time system by automatically converting serial codes into the symbolic representation of their data flow. We show how the compiler analyzes the data flow, and demonstrate that scientifically important, dense linear algebra operations can benefit from this analysis, and deliver high performance on large scale platforms.

show abstract

DAGuE: A generic distributed DAG engine for High Performance Computing

Cited by 201 publications

References 27 publications

DuctTeip: An efficient programming model for distributed task-based parallel computing

DuctTeip: An efficient programming model for distributed task-based parallel computing

DF‐DTM: Dynamic Task Memoization and reuse in dataflow

From Serial Loops to Parallel Execution on Distributed Systems

Contact Info

Product

Resources

About