A comparative performance study of common and popular task‐centric programming frameworks

Podobas, Artur; Brorsson, Mats; Faxén, Karl-Filip

doi:10.1002/cpe.3186

Cited by 11 publications

(6 citation statements)

References 38 publications

(67 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Discussionmentioning

confidence: 99%

“…Also, other computer platforms may not exhibit the pronounced non-uniform memory access (NUMA) effects common to the AMD Magny-Cours architecture, and we may expect the parallel performance of SpAMM omp on those platforms to show improved scaling. Finally, it is known that the runtime has significant impact on the performance of SpAMM-like workloads [117], and other programming frameworks might lead to improved parallel scaling. These satisfactory results follow from over-decomposition of the three-dimensional convolution space, relative to conventional methods that involve decomposition in one or two dimensions, and from runtime systems that support the irregular task parallelism inherent in the generalized N -Body solvers framework.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Solvers for $\mathcal{O} (N)$ Electronic Structure in the Strong Scaling Limit

Bock

Challacombe

Kalé

2016

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

We present a hybrid OpenMP/Charm++ framework for solving the O(N ) Self-Consistent-Field eigenvalue problem with parallelism in the strong scaling regime, P N , where P is the number of cores, and N a measure of system size, i.e. the number of matrix rows/columns, basis functions, atoms, molecules, etc. This result is achieved with a nested approach to Spectral Projection and the Sparse Approximate Matrix Multiply [Bock and Challacombe, SIAM J. Sci. Comput. 35 C72, 2013], and involves a recursive, task-parallel algorithm, often employed by generalized N -Body solvers, to occlusion and culling of negligible products in the case of matrices with decay. Employing classic technologies associated with generalized N -Body solvers, including over-decomposition, recursive task parallelism, orderings that preserve locality, and persistence-based load balancing, we obtain scaling beyond hundreds of cores per molecule for small water clusters ([H 2 O] N , N ∈ {30, 90, 150}, P/N ≈ {819, 273, 164}) and find support for an increasingly strong scalability with increasing system size N .

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Solvers for $\mathcal{O} (N)$ Electronic Structure in the Strong Scaling Limit

Bock

Challacombe

Kalé

2016

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

show abstract

“…A benchmarking study of several versions of the three shared memory systems mentioned above, which includes an investigation of task granularity, is presented in [6]. The authors find that some parallel systems handle finer granularity tasks better or worse than others, and that this also depends on the type of workload being benchmarked.…”

Section: Tiny Tasks In Shared Memory Clustersmentioning

confidence: 99%

“…Related measurements have been done before, either to develop practical guidance for improving cluster performance [2], [3], or in evaluating distributed schedulers that could support larger degrees of parallelism [5]. In [6] the effects of task granularity, and flat vs recursive task spawning, on performance are investigated for several shared-memory task-centric parallel systems. In our case we focus on statistically principled experiments that use tasks with service times drawn from controlled distributions.…”

Section: Introductionmentioning

confidence: 99%

“…The simplest of these is to partition jobs into a larger number of tasks than there are servers, k > l. A common guideline for Spark systems is that the number of tasks should be about three times the number of servers, i.e., k ≈ 3l [2], or optimized through trial and error [3]. In shared-memory systems task granularity is often discussed in terms of a target task duration [4], [6]. Some researchers have proposed even more extreme task granularity, k ≫ l, coining the term "tiny tasks" [39].…”

Section: Introduction To Tiny Tasksmentioning

confidence: 99%

See 1 more Smart Citation

The Tiny-Tasks Granularity Trade-Off: Balancing Overhead Versus Performance in Parallel Systems

Bora

Walker

Fidler

2023

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Models of parallel processing systems typically assume that one has l workers and jobs are split into an equal number of k = l tasks. Splitting jobs into k > l smaller tasks, i.e. using "tiny tasks", can yield performance and stability improvements because it reduces the variance in the amount of work assigned to each worker, but as k increases, the overhead involved in scheduling and managing the tasks begins to overtake the performance benefit. We perform extensive experiments on the effects of task granularity on an Apache Spark cluster, and based on these, developed a four-parameter model for task and job overhead that, in simulation, produces sojourn time distributions that match those of the real system. We also present analytical results which illustrate how using tiny tasks improves the stability region of split-merge systems, and analytical bounds on the sojourn and waiting time distributions of both split-merge and single-queue fork-join systems with tiny tasks. Finally we combine the overhead model with the analytical models to produce an analytical approximation to the sojourn and waiting time distributions of systems with tiny tasks which include overhead. We also perform analogous tiny-tasks experiments on a hybrid multi-processor shared memory system based on MPI and OpenMP which has no loadbalancing between nodes. Though no longer strict analytical bounds, our analytical approximations with overhead matched both the Spark and MPI/OpenMP experimental results very well.

show abstract