Group-Wise Performance Evaluation of Processor Co-allocation in Multi-cluster Systems

Abstract

Abstract. Performance evaluation in multi-cluster processor co-allocation -like in many other parallel job scheduling problems-is mostly done by computing the average metric value for the entire job stream. This does not give a comprehensive understanding of the relative performance of the different jobs grouped by their characteristics. It is however the characteristics that affect how easy/hard jobs are to schedule. We, therefore, do not get to understand scheduler performance at job type level. In this paper, we study the performance of multi-cluster processor co-allocation for different job groups grouped by their size, components and widest component. We study their relative performance, sensitivity to parameters and how their performance is affected by the heuristics used to break them up into components. We show that the widest component us characteristic that most affects job schedulability. We also show that to get better performance, jobs should be broken up in such a way that the width of the widest component is minimized.

Performance and Fairness for Users in Parallel Job Scheduling

“…The basic idea is that each job "deserves" 1/n t of the resources, where n t is the number of jobs present in the system at the given time t. The "unfairness" is computed by comparing the resources consumed by a job with the resources deserved by the job. An overview of existing techniques including discussion of their suitability can be found in [26].…”

“…Clearly, this reduce the core problem of EASY backfilling where jobs close to but not yet at the head of the queue can be significantly delayed. The price paid is that the number of jobs that can utilize existing gaps 4 is reduced, implying that more gaps are left unused in Conservative backfilling than in EASY backfilling [26]. Still, both approaches lead to significant performance improvements compared to FCFS [26,7].…”

Resource Provisioning in SLA-Based Cluster Computing

“…Feitelson et al [8] and Yom-Tov and Aridor [19] have studied resource provisioning for job scheduling in heterogeneous server clusters. Ngubiri and Vlient [14] discussed a processor provisioning problem in multi-cluster systems. Bucur [3], Bucer and Epema [4], and Jones [11] have considered the problem of resource provisioning for Distributed ASCI Supercomputer (DAS).…”

Characteristics of Fairness Metrics and Their Effect on Perceived Scheduler Effectiveness

“…This is because starved and favored jobs can easily be identified. Ngubiri and van Vliet [11] [14] show that schedulability is highly dependant on job characteristics. Different job groups (grouped by size, number of components and width of widest component) have different performances.…”

“…Sometimes, however, the a decision made on one job can have an effect that goes far deep into the queue [22]. Studies in [11] actually show that even with no consideration of job run-times (like in FPFS), all jobs achieve a net benefit.…”