Implementing multiprocessor scheduling disciplines

Parsons, Eric W.; Sevcik, Kenneth C.

doi:10.1007/3-540-63574-2_21

Cited by 29 publications

(19 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…If better knowledge of job service times than queue identities is available, then it is best to try to activate the jobs in order of increasing expected remaining service time [63]. If the service times are known to be highly variable, but the service times of individual jobs cannot be predicted in advance, then the discipline that executes the job with least acquired service first is best because it emulates the behavior of least expected remaining work first.…”

Section: Recommendationmentioning

confidence: 98%

Theory and practice in parallel job scheduling

Feitelson

Rudolph

Schwiegelshohn

et al. 1997

Lecture Notes in Computer Science

Self Cite

301

204

View full text Add to dashboard Cite

Abstract. The scheduling of jobs on parallel supercomputer is becomhag the subject of much research. However, there is concern about the divergence of theory and practice. We review theoretical research in this area, and recommendations based on recent results. This is contrasted with a proposal for standard interfaces among the components of a scheduling system, that has grown from requirements in the field.

show abstract

Section: Recommendationmentioning

confidence: 98%

Theory and practice in parallel job scheduling

Feitelson

Rudolph

Schwiegelshohn

et al. 1997

Lecture Notes in Computer Science

Self Cite

301

204

View full text Add to dashboard Cite

show abstract

“…For these, when source is not available, one could start with the ideas concerning non-invasive frontends described in Chapter 5.9.2. Parsons and Sevcik [1997] discuss extensions to the lsf clustering system [Platform, 2003]. They found that building on this commercial sys-· 231 tem was straightforward.…”

Section: Extensibility Of Existing Systemsmentioning

confidence: 99%

Cluster scheduling for explicitly-speculative tasks

Petrou

Ganger

Gibson

2004

Proceedings of the 18th Annual International Conference on Supercomputing

View full text Add to dashboard Cite

Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. AbstractA process scheduler on a shared cluster, grid, or supercomputer that is informed which submitted tasks are possibly unneeded speculative tasks can use this knowledge to better support increasingly prevalent user work habits, lowering user-visible response time, lowering user costs, and increasing resource provider revenue.Large-scale computing often consists of many speculative tasks (tasks that may be canceled) to test hypotheses, search for insights, and review potentially finished products. For example, speculative tasks are issued by bioinformaticists comparing dna sequences, computer graphics artists rendering scenes, and computer researchers studying caching. This behaviorexploratory searches and parameter studies, made more common by the costeffectiveness of cluster computing -on existing schedulers without speculative task support results in a mismatch of goals and suboptimal scheduling. Users wish to reduce their time waiting for needed task output and the amount they will be charged for unneeded speculation, making it unclear to the user how many speculative tasks they should submit. This thesis introduces 'batchactive' scheduling (combining batch and interactive characteristics) to exploit the inherent speculation in common application scenarios. With a batchactive scheduler, users submit explicitlylabeled batches of speculative tasks exploring ambitious lines of inquiry, and users interactively request task outputs when these outputs are found to be needed. After receiving and considering an output for some time, a user decides whether to request more outputs, cancel tasks, or disclose new speculative tasks. Users are encouraged to disclose more computation because batchactive scheduling intelligently prioritizes among speculative and non-speculative tasks, providing good wait-time-based metrics, and because batchactive scheduling employs an incentive pricing mechanism which charges for only requested task outputs (i.e., unneeded speculative tasks are not charged), providing better cost-based metrics for users. These aspects can lead to higher billed server utilization, encouraging batchactive adoption by resource providers organized as either cost-or profit-centers. vi · Cluster sche...

show abstract

“…The schedulers used are Load-Leveler, LSF, PBS, or NQS. These schedulers typically only support run-to-completion (no preemption) [18].…”

Section: The Task Assignment Problemmentioning

confidence: 99%

Evaluation of Task Assignment Policies for Supercomputing Servers: The Case for Load Unbalancing and Fairness

Schroeder¹

2000

View full text Add to dashboard Cite

While the MPP is still the most common architecture in supercomputer centers today, a simpler and cheaper machine configuration is growing increasingly common. This alternative setup may be described simply as a collection of multiprocessors or a distributed server system. This collection of multiprocessors is fed by a single common stream of jobs, where each job is dispatched to exactly one of the multiprocessor machines for processing. The biggest question which arises in such distributed server systems is what is a good policy for assigning jobs to host machines. Many task assignment policies have been proposed, but not systematically evaluated under supercomputing workloads. In this paper we start by comparing existing task assignment policies using a trace-driven simulation under supercomputing workloads. We use analysis to validate our results and to provide intuition. We find that while the performance of supercomputing servers varies widely with the task assignment policy, none of the above policies perform as well as we would like. We observe that all task assignment policies proposed thus far aim to balance load among the hosts. We propose a policy which purposely unbalances load among the hosts, yet, counter-to-intuition, is also fair in that it achieves the same expected slowdown for all jobs -thus no jobs are biased against. We evaluate this policy again using both trace-driven simulation and analysis. We find that the performance of the load unbalancing policy is significantly better than the best of those policies which balance load.

show abstract

Implementing multiprocessor scheduling disciplines

Cited by 29 publications

References 25 publications

Theory and practice in parallel job scheduling

Theory and practice in parallel job scheduling

Cluster scheduling for explicitly-speculative tasks

Evaluation of Task Assignment Policies for Supercomputing Servers: The Case for Load Unbalancing and Fairness

Contact Info

Product

Resources

About