An improved two-step algorithm for task and data parallel scheduling in distributed memory machines

Bansal, Sandeep; Kumar, Pramod; Singh, Kuldip

doi:10.1016/j.parco.2006.08.004

Cited by 39 publications

(58 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Paradigm compiler proposed a two step allocation and scheduling approach [39], using convex programming to decide how many processors to allocate to a task. Bansal et al proposed a two-step Modified Critical Path and Area-based (MCPA) scheduling heuristic to balance processor allocation and task assignment [40]. Another study employed performance modeling in workflow scheduling on Grid resources [41].…”

Section: Related Workmentioning

confidence: 99%

Machine learning based online performance prediction for runtime parallelization and task scheduling

Singh

et al. 2009

2009 IEEE International Symposium on Performance Analysis of Systems and Software

View full text Add to dashboard Cite

Abstract-With the emerging many-core paradigm, parallel programming must extend beyond its traditional realm of scientific applications. Converting existing sequential applications as well as developing next-generation software requires assistance from hardware, compilers and runtime systems to exploit parallelism transparently within applications. These systems must decompose applications into tasks that can be executed in parallel and then schedule those tasks to minimize load imbalance. However, many systems lack a priori knowledge about the execution time of all tasks to perform effective load balancing with low scheduling overhead.In this paper, we approach this fundamental problem using machine learning techniques first to generate performance models for all tasks and then applying those models to perform automatic performance prediction across program executions. We also extend an existing scheduling algorithm to use generated task cost estimates for online task partitioning and scheduling. We implement the above techniques in the pR framework, which transparently parallelizes scripts in the popular R language, and evaluate their performance and overhead with both a realworld application and a large number of synthetic representative test scripts. Our experimental results show that our proposed approach significantly improves task partitioning and scheduling, with maximum improvements of 21.8%, 40.3% and 22.1% and average improvements of 15.9%, 16.9% and 4.2% for LMM (a real R application) and synthetic test cases with independent and dependent tasks, respectively.

show abstract

Section: Related Workmentioning

confidence: 99%

Machine learning based online performance prediction for runtime parallelization and task scheduling

Singh

et al. 2009

2009 IEEE International Symposium on Performance Analysis of Systems and Software

View full text Add to dashboard Cite

show abstract

“…As discussed earlier, most of the scheduling algorithms for mixed parallel applications on homogeneous clusters proceed in two phases [3], [4], [5], [6]. A prominent algorithm is CPA (Critical Path and Area-based scheduling) [4], which aims at finding the best compromise between two quantities.…”

Section: Related Workmentioning

confidence: 99%

“…Two algorithms address this limitation. MCPA [3] limits processor allocations to ensure that all the tasks in a level of the application DAG can be executed concurrently. This algorithm is applicable only to very regular DAGs.…”

Section: Related Workmentioning

confidence: 99%

“…CONCLUSION Two-step algorithms for scheduling mixed-parallel applications on clusters structured as a task graph whose nodes are data-parallel computations have been developed [3], [4], [5], [6], [7]. However these algorithms separate the determination of the number of processors to allocate to each task from the process that decides on which processors of the cluster each task is to be executed.…”

Section: Comparing Tuned Rats To Hcpamentioning

confidence: 99%

“…Most clusters consist of identical compute nodes (at least when they are initially put in production) and thus the question of scheduling mixedparallel applications on homogeneous platforms has been studied by many researchers. Several practical scheduling algorithms based on heuristics have been proposed in the literature [3], [4], [5], [6] that proceed in two steps. In a first step, the algorithm decides how many processors should be allocated to each task, while in step two, the algorithm uses a list scheduling approach to map tasks to sets of processors.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Redistribution aware two-step scheduling for mixed-parallel applications

Hunold

Rauber

Suter³

2008

2008 IEEE International Conference on Cluster Computing

View full text Add to dashboard Cite

Abstract-Applications raising in many scientific fields exhibit both data and task parallelism that have to be exploited efficiently. A classic approach is to structure those applications by a task graph whose nodes represent parallel computations. Scheduling such mixed-parallel applications is challenging even on a single homogeneous platform, such as a cluster. Most of the mixed-parallel application scheduling algorithms rely on two decoupled steps: allocation and mapping. This separation can induce unnecessary or costly data redistributions that have an impact on the overall performance. This is particularly true for data intensive applications. In this paper, we propose an original approach in which the allocations determined in the first step can be adapted during the second step in order to minimize the impact of these data redistributions. Two redistribution aware mapping strategies are detailed and a study of their impact on the schedule length is proposed through a comparison with an efficient two step algorithm over a broad range of experimental scenarios.

show abstract

Optimizing layer‐based scheduling algorithms for parallel tasks with dependencies

Kunis

Rünger

2010

Concurrency and Computation

View full text Add to dashboard Cite

SUMMARYProgramming with parallel tasks leads to task graphs with dependencies representing a parallel program. Scheduling algorithms are employed to find an efficient execution order of the parallel tasks. A large variety of scheduling algorithms exist, including layer-based scheduling algorithms for homogeneous target platforms that build consecutive layers of independent parallel tasks and schedule each layer separately. Although these scheduling algorithms provide good results in terms of scheduling algorithm runtime and schedule execution time, the resulting schedules leave room for optimization. This article proposes an optimization for arbitrary layer-based scheduling algorithms, which is called Move-blocks algorithm. Given a layer-based schedule of the parallel tasks, this algorithm moves blocks of parallel tasks into preceding layers in order to reduce the overall execution time of a task-based application. Suitable blocks of parallel tasks are identified by the algorithm Find-blocks, which is employed together with the Move-blocks algorithm. The algorithm Move-blocks is applied to four well-known scheduling algorithms. A detailed evaluation for a wide range of test cases is given.

show abstract

An improved two-step algorithm for task and data parallel scheduling in distributed memory machines

Cited by 39 publications

References 18 publications

Machine learning based online performance prediction for runtime parallelization and task scheduling

Machine learning based online performance prediction for runtime parallelization and task scheduling

Redistribution aware two-step scheduling for mixed-parallel applications

Optimizing layer‐based scheduling algorithms for parallel tasks with dependencies

Contact Info

Product

Resources

About