An Adaptive Skeletal Task Farm for Grids

Concurrency and Computation

2010

Self Cite

SUMMARYAlgorithmic skeletons abstract commonly used patterns of parallel computation, communication, and interaction. Based on the algorithmic skeleton concept, structured parallelism provides a high-level parallel programming technique that allows the conceptual description of parallel programs while fostering platform independence and algorithm abstraction. This work presents a methodology to improve skeletal parallel programming in heterogeneous distributed systems by introducing adaptivity through resource awareness. As we hypothesise that a skeletal program should be able to adapt to the dynamic resource conditions over time using its structural forecasting information, we have developed adaptive structured parallelism (ASPARA). ASPARA is a generic methodology to incorporate structural information at compilation into a parallel program, which will help it to adapt at execution. ASPARA comprises four phases: programming, compilation, calibration, and execution. We illustrate the feasibility of this approach and its associated performance improvements using independent case studies based on two algorithmic skeletons-the task farm and the pipeline-evaluated in a non-dedicated heterogeneous multi-cluster system.

supporting

confidence: 75%

Adaptive structured parallelism for distributed heterogeneous architectures: a methodological approach with pipelines and farms

Concurrency and Computation

2010

Self Cite

“…Based on previous experiences with skeletons on geographically dispersed grids [7], where we empirically learned the costly implications of the inherent synchronisation in collectives, we have based this design on explicit send-receive pairing. Internally, each stage is composed by a MPI Recv call, the invocation to the f function, and a MPI Send call.…”

Section: Methodsmentioning

confidence: 99%

Towards Fully Adaptive Pipeline Parallelism for Heterogeneous Distributed Environments

Parallel and Distributed Processing and Applications

2006

Self Cite

This work describes an adaptive parallel pipeline skeleton which maps pipeline stages to the best processors available in the system and clears dynamically emerging performance bottlenecks at run-time by re-mapping affected stages to other processors. It is implemented in C and MPI and evaluated on a non-dedicated heterogeneous Linux cluster. We report upon the skeleton's ability to respond to an artificially generated variation in the background load across the cluster.

“…This work significantly extends our initial findings on single-round scheduling (González-Vélez 2006) and parameterisable skeletal task farms (González-Vélez 2005), by providing a comprehensive statistical online framework to automatically schedule divisible workloads based on the dispersion of the participating nodes and size of the workload. Being application-agnostic and parameterisable, our approach addresses the multi-round scheduling case by defining an installment factor which dynamically quantifies the number of rounds using the number of tasks in the workload and the system circumstances.…”

Section: Contribution and Structurementioning

confidence: 60%

Adaptive statistical scheduling of divisible workloads in heterogeneous systems

2009

J Sched

Self Cite

This article presents a statistical approach to the scheduling of divisible workloads. Structured as a task farm with different scheduling modes including adaptive single and multi-round scheduling, this novel divisible load theory approach comprises two phases, calibration and execution, which dynamically adapt the installment size and number. It introduces the concept of a generic installment factor based on the statistical dispersion of the calibration times of the participating nodes, which allows automatic determination of the number and size of the workload installments. Initially, the calibration ranks processors according to their fitness and determines an installment factor based on how different their execution times are. Subsequently, the execution iteratively distributes the workload according to the processor fitness, which is continuously re-assessed throughout the program execution. Programmed as an adaptive algorithmic skeleton, our task farm has been successfully evaluated for single-round scheduling and generic multi-round scheduling using a computational biology parameter-sweep in a nondedicated multi-cluster system.