Performance and Fairness for Users in Parallel Job Scheduling

Abstract

Abstract.In this work we analyze the performance of scheduling algorithms with respect to fairness. Existing works frequently consider fairness as a job related issue. In our work we analyze fairness with respect to different users of the system as this is a very important real-life problem. First, we discuss how fair are selected popular scheduling algorithms with respect to different users of the system. Next, we present an extension to the well known Conservative backfilling algorithm. Instead of "ad hoc" decisions, the schedule is now created subject to evaluation and optimization. Notably, the fairness is considered as an important metric, which accompanies standard performance related metrics such as slowdown or wait time. To achieve that, an inclusion of fairness as an optimization criterion is proposed. The new extension improves the performance and fairness of Conservative backfilling with respect to other classical techniques such as FCFS, EASY backfilling or aggressive backfilling without reservations.

Don’t Hurry Be Happy: A Deadline-Based Backfilling Approach

“…This may come from a dierent resource fragmentation that appears as we skip some deadline-driven jobs while building L tmp . We thus add an extra step (lines [22][23][24][25][26][27][28][29][30][31][32][33][34][35], in which we move from L 1 to L tmp all the deadline-driven jobs submitted before the last job in L tmp unable to respect its deadline, before recomputing all the allocations. The next step consists in determining which jobs in L tmp can start their execution in this scheduling round (line 38).…”

“…In [22,23], the authors rely on meta-heuristics, i.e., tabu-search and random selection, to periodically reorganize the schedules. While these modications improve the average wait time and stretch, they do not preserve one of the most interesting feature of CBF which is to provide an upper bound of job completion time on submission.…”

A Periodic Portfolio Scheduler for Scientific Computing in the Data Center

“…Especially when servicing the demanding workloads typical of scientific computing [1,2], these data centers need efficient algorithms for scheduling their users' workloads on the data center resources. Many existing scheduling algorithms have already addressed specific workload properties [3,4] and types of applications [5][6][7][8], but data centers still rely on (expensive) human system administrators to select a scheduling algorithm and configure it appropriately. Moreover, the selection process is made significantly more difficult by changing workloads due to technology transitions (e.g., the use of virtualization and new networking architectures), and by new customers starting to use data centers as Infrastructure-as-a-Service clouds.…”

Planning and Optimization in TORQUE Resource Manager

“…This paper builds on the top of our earlier "theoretical" works [6,5] which have proposed new methods for efficient use of metaheuristic algorithms for multi-criteria scheduling in grids. However, instead of a real resource manager, these works only used a simulator while considering simplified problem models.…”

“…However, instead of a real resource manager, these works only used a simulator while considering simplified problem models. For example, job memory requirements were ignored [5] and precise job runtimes were used instead of more realistic inaccurate estimates [6].…”