Task‐based FMM for heterogeneous architectures

Agullo, Emmanuel; Bramas, Bérenger; Coulaud, Olivier; Darve, Eric; Messner, Matthias; Takahashi, Tōru

doi:10.1002/cpe.3723

Cited by 38 publications

(42 citation statements)

References 42 publications

(60 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…H etero P rio has been proposed in the context of task‐based runtime systems responsible for allocating tasks onto heterogeneous nodes typically consisting of a few CPUs and GPUs …”

Section: Presentation Of Heteropriomentioning

confidence: 99%

“…HETEROPRIO has been proposed in the context of task-based runtime systems responsible for allocating tasks onto heterogeneous nodes typically consisting of a few CPUs and GPUs. 17 Historically, most systems use scheduling strategies inspired by the well-known HEFT algorithm: tasks are ordered by priorities (which are computed offline) and the highest priority ready task is allocated onto the resource that is expected to complete it first, given the expected transfer times of its input data and the expected processing time of this task on this resource. These systems have shown some limits 13 in strongly heterogeneous and unrelated systems, which is the typical case of nodes consisting of both CPUs and GPUs.…”

Section: Affinity-based Schedulingmentioning

confidence: 99%

See 1 more Smart Citation

Fast approximation algorithms for task‐based runtime systems

Beaumont

Eyraud-Dubois

Kumar

2018

Concurrency and Computation

View full text Add to dashboard Cite

Summary In High Performance Computing, heterogeneity is now the norm with specialized accelerators like GPUs providing efficient computational power. Resulting complexity led to the development of task‐based runtime systems, where complex computations are described as task graphs, and scheduling decisions are made at runtime to perform load balancing between all the resources of the platforms. Developing good scheduling strategies, even at the scale of a single node, and analyzing them both theoretically and in practice are expected to have a very high impact on the performance of current HPC systems. The special case of 2 kinds of resources, typically CPUs and GPUs, is already of great practical interest. The scheduling policy HeteroPrio has been proposed in the context of fast multipole computations (FMM) and has been extended to general task graphs with very promising results. In this paper, we provide a theoretical study of the performance of HeteroPrio, by proving approximation bounds compared to the optimal schedule, both in the case of independent tasks and in the case of general task graphs. Interestingly, our results establish that spoliation (a technique that enables resources to restart uncompleted tasks on another resource) is enough to prove bounded approximation ratios for a list scheduling algorithm on two unrelated resources, which is known to be impossible otherwise. This result holds true both for the independent and dependent tasks graphs. Additionally, we provide an experimental evaluation of HeteroPrio on real task graphs from dense linear algebra computation, which establishes its strong performance in practice.

show abstract

“…H etero P rio has been proposed in the context of task‐based runtime systems responsible for allocating tasks onto heterogeneous nodes typically consisting of a few CPUs and GPUs …”

Section: Presentation Of Heteropriomentioning

confidence: 99%

Section: Affinity-based Schedulingmentioning

confidence: 99%

Fast approximation algorithms for task‐based runtime systems

Beaumont

Eyraud-Dubois

Kumar

2018

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…By providing the abstraction of its programming layer, OPENMP makes it possible to get the best of both worlds. In the future, we intend to explore the coupling of OPENMP with task-based runtime systems further, in particular focusing on the support for heterogeneous CPU+accelerator platforms that could be illustrated with a task-based FMM for heterogeneous machines [26].…”

Section: Resultsmentioning

confidence: 99%

Bridging the Gap Between OpenMP and Task-Based Runtime Systems for the Fast Multipole Method

Agullo

Aumage

Bramas

et al. 2017

IEEE Trans. Parallel Distrib. Syst.

Self Cite

View full text Add to dashboard Cite

With the advent of complex modern architectures, the low-level paradigms long considered sufficient to build High Performance Computing (HPC) numerical codes have met their limits. Achieving efficiency, ensuring portability, while preserving programming tractability on such hardware prompted the HPC community to design new, higher level paradigms while relying on runtime systems to maintain performance. However, the common weakness of these projects is to deeply tie applications to specific expert-only runtime system APIs. The OpenMP specification, which aims at providing common parallel programming means for shared-memory platforms, appears as a good candidate to address this issue thanks to the latest task-based constructs introduced in its revision 4.0. The goal of this paper is to assess the effectiveness and limits of this support for designing a high-performance numerical library, ScalFMM, implementing the fast multipole method (FMM) that we have deeply re-designed with respect to the most advanced features provided by OpenMP 4. We show that OpenMP 4 allows for significant performance improvements over previous OpenMP revisions on recent multicore processors and that extensions to the 4.0 standard allow for strongly improving the performance, bridging the gap with the very high performance that was so far reserved to expert-only runtime system APIs.Index Terms-high performance computing, fast multipole method, runtime system, OpenMP, compiler, parallel programming model, priority, commutativity, multicore architecture ! Algorithm 5: tb-omp4#task#dep scheme with OPENMP 4.0 directives 1 function FMM(tree, kernel) 2 #pragma omp parallel 3 #pragma omp single 4 // Near-field 5 P2P_taskdep(tree, kernel); 18 foreach cell cl in tree.cells[level] do 19 #pragma omp task depend(inout:cl.multipole) \\ 20 depend(in:tree.getChildren(cl.mindex, level).multipole) 21 kernel.M2M(cl.multipole, tree.getChildren(cl.mindex, level).multipole); STARPU directives 1 function FMM(tree, kernel) 2 // Near-field 3 P2P_starpu(tree, kernel);

show abstract

“…Most of these tools support a core part of task-based RS, such as creating a graph of tasks (even if it is implemented differently) where tasks can read or write data. However, scheduling is an important factor in the performance (Agullo et al, 2016a), and few of these RS's propose a way to create a scheduler easily without having to go inside the RS's code. Moreover, specific features provide mechanisms to increase the degree of parallelism.…”

Section: Task-based Parallelizationmentioning

confidence: 99%

Increasing the degree of parallelism using speculative execution in task-based runtime systems

Bramas

2019

PeerJ Computer Science

View full text Add to dashboard Cite

Task-based programming models have demonstrated their efficiency in the development of scientific applications on modern high-performance platforms. They allow delegation of the management of parallelization to the runtime system (RS), which is in charge of the data coherency, the scheduling, and the assignment of the work to the computational units. However, some applications have a limited degree of parallelism such that no matter how efficient the RS implementation, they may not scale on modern multicore CPUs. In this paper, we propose using speculation to unleash the parallelism when it is uncertain if some tasks will modify data, and we formalize a new methodology to enable speculative execution in a graph of tasks. This description is partially implemented in our new C++ RS called SPETABARU, which is capable of executing tasks in advance if some others are not certain to modify the data. We study the behavior of our approach to compute Monte Carlo and replica exchange Monte Carlo simulations. Subjects Distributed and Parallel ComputingHow to cite this article Bramas B. 2019. Increasing the degree of parallelism using speculative execution in task-based runtime systems. PeerJ Comput. Sci. 5:e183 http://doi.org/10.7717/peerj-cs.183 Bramas (2019), PeerJ Comput. Sci., DOI 10.7717/peerj-cs.183 2/25 Bramas (2019), PeerJ Comput. Sci., DOI 10.7717/peerj-cs.183 3/25 ALGORITHM 2: Replica Exchange Monte Carlo (parallel tempering) simulation algorithm. 1 function REMC(domains[N], temperature[N]) 2 // Compute energy (particle to particle interactions) 3 for s from 1 to N do 4 energy[s] ← compute_energy(domains[s]) 5 end 6 // Iterate for a given number of iterations 7 for iter from 1 to NB_LOOPS_REMC do 8 for s from 1 to N do 9 // Compute usual MC for each simulation 10 MC_Core(domains[s], temperature[s], energy[s]) 11 end 12 // Compare based on a given strategy 13 for s in exchange_list(iter) do 14 // Use the energy difference between s and s+1 to decide to exchange them 15 if random_01() ≤ metropolis(energy[s] -energy[s+1], temperatures[s]) then 16 swap(domains[s], domains[s+1]) 17 swap(energy[s], energy[s+1]) 18 end 19 end 20 end Bramas (2019), PeerJ Comput. Sci., DOI 10.7717/peerj-cs.183 6/25 Thachuk C, Shmygelska A, Hoos HH. 2007. A replica exchange Monte Carlo algorithm for protein folding in the HP model. BMC Bioinformatics 8(1):342 Nikolopoulos DS. 2018. A taxonomy of task-based parallel programming technologies for highperformance computing. The Journal of Supercomputing 74(4):1422-1434. Tillenius M. 2015. Superglue: a shared memory framework using data versioning for dependency-aware task-based parallelization. SIAM Journal on Scientific Computing 37(6):C617-C642 X. 2013. Parallel metropolis coupled Markov chain Monte Carlo for isolation with migration model. Applied Mathematics & Information Sciences 7(1L):219-224 DOI 10.12785/amis/071L30.

show abstract

Task‐based FMM for heterogeneous architectures

Cited by 38 publications

References 42 publications

Fast approximation algorithms for task‐based runtime systems

Fast approximation algorithms for task‐based runtime systems

Bridging the Gap Between OpenMP and Task-Based Runtime Systems for the Fast Multipole Method

Increasing the degree of parallelism using speculative execution in task-based runtime systems

Contact Info

Product

Resources

About