The performance of bags-of-tasks in large-scale distributed systems

Iosup, Alexandru; Sonmez, Ozan; Anoep, Shanny; Epema, Dick

doi:10.1145/1383422.1383435

Cited by 122 publications

(129 citation statements)

References 30 publications

Supporting

Mentioning

117

Contrasting

Unclassified

Order By: Relevance

“…Similarly this has been done for the small (120,000,000) and large (480,000,000) categories. The workload for a task will have a normal distribution with a mean of 2.73 and a standard deviation of 6.1 following the average task runtime indicated in [13]. The durations were chosen because [11] gives the average time to run a task in a group submission/job as 14,181s and [12,10] indicates that most runtimes are shown to between a fraction of a minute and 1,000 min (1,000min = 16.6hrs).…”

Section: Methodsmentioning

confidence: 99%

“…The durations were chosen because [11] gives the average time to run a task in a group submission/job as 14,181s and [12,10] indicates that most runtimes are shown to between a fraction of a minute and 1,000 min (1,000min = 16.6hrs). The runtime variability between tasks belonging to the same BoT [13] is indicated to follow a Weibull distribution with a shape of 2.05 and a scale parameter of 12.25 hence this variability will also be accounted for.…”

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

“…A Weibull distribution will be chosen with a shape parameter set to 1.76 as per [13], the scale parameter will be adjusted in a fashion that creates the appropriate range desired. Iosup et al [13] give an average BoT size in their traces as been between 5 and 50, while the maximum BoT size can be on the order of thousands. The authors in [10] indicate that the average number of tasks in a bag in various different Grid traces investigated ranges between 2 and The inter-arrival time (i.e.…”

Section: Methodsmentioning

confidence: 99%

“…A substantial part of Grid workloads are formed from Bag of Task/Parameter sweep based applications [13,10]. These are formed by sets of tasks that execute independently of one another without communication and are typically considered to be "embarrassingly parallel" [19].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A Grid Broker Pricing Mechanism for Temporal and Budget Guarantees

Kavanagh

Djemame

2011

Computer Performance Engineering

View full text Add to dashboard Cite

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Grid Broker Pricing Mechanism for Temporal and Budget Guarantees

Kavanagh

Djemame

2011

Computer Performance Engineering

View full text Add to dashboard Cite

show abstract

Coordinated rescheduling of Bag‐of‐Tasks for executions on multiple resource providers

Netto

Buyya

2011

Concurrency and Computation

View full text Add to dashboard Cite

Abstract-Metaschedulers can distribute parts of a Bag-of-Tasks (BoT) application among various resource providers in order to speed up its execution. The expected completion time of the user application is then calculated based on the run time estimates of all applications running and waiting for resources. However, due to inaccurate run time estimates, initial schedules are not those that provide users with the earliest completion time. These estimates increase the time distance between the first and last tasks of a BoT application, which increases average user response time, especially in multi-provider environments. This paper proposes a coordinated rescheduling algorithm to handle inaccurate run time estimates when executing BoT applications in multi-provider environments. The coordinated rescheduling defines which tasks can have start time updated based on the expected completion time of the entire BoT application. We have also evaluated the impact of system-generated run time estimates to schedule BoT applications on multiple providers. We performed experiments using simulations and a real distributed platform, Grid'5000. From our experiments, we obtained reductions of up to 5% and 10% for response time and slowdown metrics respectively by using coordinated rescheduling over a traditional rescheduling solution. Moreover, coordinated rescheduling requires little modification of existing scheduling systems. System-generated predictions, on the other hand, are more complex to be deployed and may not reduce response times as much as when using coordinated rescheduling.

show abstract

A bi‐criteria scheduling process with CoS support on grids and clouds

Kloh

Schulze

Pinto

et al. 2011

Concurrency and Computation

View full text Add to dashboard Cite

This paper presents a workflow scheduling model, which is based on a hybrid bi-criteria scheduling algorithm based on a class of service approach. The proposed model is based on the analysis of several related works on resource scheduling, trying to incorporate the most relevant aspects considered in these works, thus covering some shortcomings pointed out by them. The proposed model aims to optimize the criteria chosen by the users, based on their specified priority order and a variation limit specified by them. In order to validate this model, a set of tests were performed comparing the performance of the proposed model with the Join the Shortest Queue scheduling algorithm. The analysis of the obtained results showed an improvement in the overall quality and also a performance gain made available to the users -as they meet the priority criteria.

show abstract

The performance of bags-of-tasks in large-scale distributed systems

Cited by 122 publications

References 30 publications

A Grid Broker Pricing Mechanism for Temporal and Budget Guarantees

A Grid Broker Pricing Mechanism for Temporal and Budget Guarantees

Coordinated rescheduling of Bag‐of‐Tasks for executions on multiple resource providers

A bi‐criteria scheduling process with CoS support on grids and clouds

Contact Info

Product

Resources

About