Fairness in parallel job scheduling

Abstract

This paper introduces a new preemptive algorithm that is well suited for fair on‐line scheduling of parallel jobs. Fairness is achieved by selecting job weights to be equal to the resource consumption of the job and by limiting the time span a job can be delayed by other jobs submitted after it. Further, the processing time of a job is not known when the job is released. It is proven that the algorithm achieves a constant competitive ratio for both the makespan and the weighted completion time for the given weight selection. Finally, the algorithm is also experimentally evaluated with the help of workload traces. Copyright © 2000 John Wiley & Sons, Ltd.

Dynamic Fractional Resource Scheduling for cluster platforms

“…An additional problem with the scheduling paradigms proposed so far is that they do not optimize a user-centric objective function; it is known that there is a sharp disconnect between user concerns (low job turn-around time, fairness) and the schedules implemented in practice [15,16]. Batch schedulers provide myriad configuration parameters by which cluster administrators can influence scheduling behaviors, but these parameters are not directly related to user-centric measures of performance.…”

Dynamic Fractional Resource Scheduling for cluster platforms

“…An additional problem with the scheduling paradigms proposed so far is that they do not optimize a user-centric objective function; it is known that there is a sharp disconnect between user concerns (low job turn-around time, fairness) and the schedules implemented in practice [15,16]. Batch schedulers provide myriad configuration parameters by which cluster administrators can influence scheduling behaviors, but these parameters are not directly related to user-centric measures of performance.…”

Scalable system scheduling for HPC and big data

“…Next we must address related work within the scheduling function. There are a number of papers that focus on comparing the queue management and scheduling algorithms within schedulers, including discussion of support for first-come, first-served (FCFS) [Schwiegelshohn and Yahyapour (1998)], fairshare [Schwiegelshohn and Yahyapour (2000)], backfill [Lifka (1995), and gang scheduling [Ousterhout (1982), Feitelson and Jette (1997)]. These papers includes comparisons of how jobs are prioritized within queues and how jobs are matched to resources to most effectively execute them.…”

A scheduling optimization technique based on reuse in spark to defend against apt attack

“…Thus, the scheduling problem [19] still exists. Common scheduling algorithms in distributed clusters are First-In First-Out (FIFO) [20] , round-robin scheduling [21] , priority scheduling [22] , and fairness scheduling [23] .…”

Dynamic Fractional Resource Scheduling for cluster platforms

“…An additional problem with the scheduling paradigms proposed so far is that they do not optimize a user-centric objective function; it is known that there is a sharp disconnect between user concerns (low job turn-around time, fairness) and the schedules implemented in practice [15,16]. Batch schedulers provide myriad configuration parameters by which cluster administrators can influence scheduling behaviors, but these parameters are not directly related to user-centric measures of performance.…”

Scalable system scheduling for HPC and big data

“…Next we must address related work within the scheduling function. There are a number of papers that focus on comparing the queue management and scheduling algorithms within schedulers, including discussion of support for first-come, first-served (FCFS) [Schwiegelshohn and Yahyapour (1998)], fairshare [Schwiegelshohn and Yahyapour (2000)], backfill [Lifka (1995), and gang scheduling [Ousterhout (1982), Feitelson and Jette (1997)]. These papers includes comparisons of how jobs are prioritized within queues and how jobs are matched to resources to most effectively execute them.…”

A scheduling optimization technique based on reuse in spark to defend against apt attack

“…Thus, the scheduling problem [19] still exists. Common scheduling algorithms in distributed clusters are First-In First-Out (FIFO) [20] , round-robin scheduling [21] , priority scheduling [22] , and fairness scheduling [23] .…”

Dynamic Fractional Resource Scheduling for cluster platforms

“…An additional problem with the scheduling paradigms proposed so far is that they do not optimize a user-centric objective function; it is known that there is a sharp disconnect between user concerns (low job turn-around time, fairness) and the schedules implemented in practice [15,16]. Batch schedulers provide myriad configuration parameters by which cluster administrators can influence scheduling behaviors, but these parameters are not directly related to user-centric measures of performance.…”

Scalable system scheduling for HPC and big data

“…Next we must address related work within the scheduling function. There are a number of papers that focus on comparing the queue management and scheduling algorithms within schedulers, including discussion of support for first-come, first-served (FCFS) [Schwiegelshohn and Yahyapour (1998)], fairshare [Schwiegelshohn and Yahyapour (2000)], backfill [Lifka (1995), and gang scheduling [Ousterhout (1982), Feitelson and Jette (1997)]. These papers includes comparisons of how jobs are prioritized within queues and how jobs are matched to resources to most effectively execute them.…”

A scheduling optimization technique based on reuse in spark to defend against apt attack

“…Thus, the scheduling problem [19] still exists. Common scheduling algorithms in distributed clusters are First-In First-Out (FIFO) [20] , round-robin scheduling [21] , priority scheduling [22] , and fairness scheduling [23] .…”