Workflow Fairness Control on Online and Non-clairvoyant Distributed Computing Platforms

Silva, Rafael Ferreira da; Glatard, Tristan; Desprez, Frédéric

doi:10.1007/978-3-642-40047-6_13

Cited by 3 publications

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We consider a non-clairvoyant, online fairness problem where the platform workload, task costs and resource characteristics are unknown and not stationary. We propose a fairness control loop which assigns task priorities based on the fraction of pending work in the workflows Ferreira da Silva, Glatard, & Desprez, 2013c). Workflow characteristics and performance on the target resources are estimated progressively, as information becomes available during the execution.…”

Section: Controlling Fairness Among Workflow Executionsmentioning

confidence: 99%

Self-Management of Operational Issues for Grid Computing

Silva

Glatard

Desprez

2015

Advances in Systems Analysis, Software Engineering, and High Performance Computing

Self Cite

View full text Add to dashboard Cite

Science gateways, such as the Virtual Imaging Platform (VIP), enable transparent access to distributed computing and storage resources for scientific computations. However, their large scale and the number of middleware systems involved in these gateways lead to many errors and faults. This chapter addresses the autonomic management of workflow executions on science gateways in an online and non-clairvoyant environment, where the platform workload, task costs, and resource characteristics are unknown and not stationary. The chapter describes a general self-management process based on the MAPE-K loop (Monitoring, Analysis, Planning, Execution, and Knowledge) to cope with operational incidents of workflow executions. Then, this process is applied to handle late task executions, task granularities, and unfairness among workflow executions. Experimental results show how the approach achieves a fair quality of service by using control loops that constantly perform online monitoring, analysis, and execution of a set of curative actions.

show abstract

Section: Controlling Fairness Among Workflow Executionsmentioning

confidence: 99%

Self-Management of Operational Issues for Grid Computing

Silva

Glatard

Desprez

2015

Advances in Systems Analysis, Software Engineering, and High Performance Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…The results indicate that the starvation problem is solved and the performance is enhanced. Moreover, to create a fairness between jobs, da Silva [3] proposed a new backfilling model to reduce fragmentat ion and to obtain better response times. At the same time, fairness between jobs is kept low and predictability remains high.…”

Section: Introductionmentioning

confidence: 99%

Enhanced Job Ranking Backfilling Based on Linear and Logarithmic Ranking Equations

Al-Najjar

Alhady

Saleh³

2019

IJIE

View full text Add to dashboard Cite

Grid system is used by many researchers' and scholars all over the world to solve the complicated and complex problems in different sciences. Job ranking backfilling is the most used model by many researchers in grid system to improve the performance of job scheduling algorithm. The model aims on serving the smallest job in the queue. As a second improvement of job backfilling, researchers proposed job ranking backfilling that serve job based on ranking equation. This paper proposes an enhance job ranking algorithm based on using linear and logarithmic ranking equations. Both proposed ranking equations used curve estimation model to predict on the variables' coefficients. By simulation and after different tests, the average results of job ranking backfilling with linear ranking equation outperform conventional job ranking backfilling with improvement equal 3.2% and 56.53% in total execution time and average waiting time, respectively. In addition, job ranking backfilling with logarithmic ranking equation shows average improvement equal 1.78% and 46.62% in total execution time and average waiting time, respectively. The results indicate that the proposed ranking equations would improve conventional job ranking backfilling in high and low demand grid system under different conditions.

show abstract

Controlling fairness and task granularity in distributed, online, non‐clairvoyant workflow executions

Silva

Glatard

Desprez

2014

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

International audienceDistributed computing infrastructures are commonly used for scientific computing, and science gateways provide complete middleware stacks to allow their transparent exploitation by end users. However, administrating such systems manually is time consuming and sub-optimal because of the complexity of the execution conditions. Algorithms and frameworks aiming at automating system administration must deal with online and non-clairvoyant conditions, where most parameters are unknown and evolve over time. We consider the problem of controlling task granularity and fairness among scientific workflows executed in these conditions. We present two self-managing loops monitoring the fineness, coarseness, and fairness of workflow executions, comparing these metrics with thresholds extracted from knowledge acquired in previous executions and planning appropriate actions to maintain these metrics to appropriate ranges. Experiments on the European Grid Infrastructure show that our task granularity control can speed up executions up to a factor of 2 and that our fairness control reduces slowdown variability by 3-7 compared with first-come, first-served. We also study the interaction between granularity control and fairness control: our experiments demonstrate that controlling task granularity degrades fairness but that our fairness control algorithm can compensate this degradation

show abstract

Workflow Fairness Control on Online and Non-clairvoyant Distributed Computing Platforms

Cited by 3 publications

References 17 publications

Self-Management of Operational Issues for Grid Computing

Self-Management of Operational Issues for Grid Computing

Enhanced Job Ranking Backfilling Based on Linear and Logarithmic Ranking Equations

Controlling fairness and task granularity in distributed, online, non‐clairvoyant workflow executions

Contact Info

Product

Resources

About