Alternatives to Coscheduling a Network of Workstations

Nagar, Seema; Banerjee, Ajit; Sivasubramaniam, Anand; Das, Chita R.

doi:10.1006/jpdc.1999.1576

Cited by 26 publications

(54 citation statements)

References 20 publications

(52 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many papers even make the expedient assumption that all parallel jobs run on all the available nodes [56,57,[65][66][67].…”

Section: Discussionmentioning

confidence: 99%

Dynamic Fractional Resource Scheduling for cluster platforms

Stillwell

2010

2010 IEEE International Symposium on Parallel &Amp; Distributed Processing, Workshops and PHD Forum (IPDPSW)

View full text Add to dashboard Cite

Experiments presented in this paper were carried out using the Grid'5000 experimental testbed, being developed under the INRIA ALADDIN development action with support from CNRS, RENATER and several Universities as well as other funding bodies (see https://www.grid5000.fr).The experiments using our prototype system were carried out on machines owned by the Deparment of Computer Science and Engineering at the University of California, San Diego with the permission of Geoffrey Voelker. System administration and much assistance were provided by Gjergji Zyba. iv ABSTRACTThis research focuses on the problem of job scheduling on homogeneous computational clusters. Clusters are widely used today for a variety of purposes, including high-performance scientific computing and Internet service hosting. While clusters may have impressive aggregate performance metrics, they are really only collections of fairly modest machines, which makes scheduling jobs for the best performance a non-trivial problem. Most clusters also need to be shared among users to amortize their start-up and maintenance costs, and ensuring that these users are treated fairly further adds to the difficulty. Existing approaches to scheduling attempt to address both of these issues, but have several limitations.We propose a novel approach, called Dynamic Fractional Resource Scheduling (DFRS), to sharing homogeneous cluster computing platforms among competing jobs.The key features of DFRS are that it leverages existing virtual machine technology in order to share resources more efficiently and it defines and optimizes a user-centric metric that captures notions of both performance and fairness. In this dissertation we explain the principles behind DFRS and its advantages over the current state of the art, develop a theoretical model of resource sharing, design heuristics to optimize the proposed metric within the given framework, implement and run simulations comparing DFRS to traditional approaches using popular and accepted performance metrics, and finally develop and test a prototype implementation based on existing technologies. Our results show that it is possible to develop heuristic algorithms that give results reasonably close to theoretical bounds for a variety of cases, that resource requirements are well within the capabilities of modern systems, and that for some scenarios DFRS can provide orders-ofmagnitude levels of improvement in performance over current approaches.

show abstract

“…Many papers even make the expedient assumption that all parallel jobs run on all the available nodes [56,57,[65][66][67].…”

Section: Discussionmentioning

confidence: 99%

Dynamic Fractional Resource Scheduling for cluster platforms

Stillwell

2010

2010 IEEE International Symposium on Parallel &Amp; Distributed Processing, Workshops and PHD Forum (IPDPSW)

View full text Add to dashboard Cite

show abstract

“…As described in [4], [11], CDC schemes can be classified by two components: message waiting action taken by processes waiting for a message and message handling action performed by the kernel when a message arrives (see Table 1). Sobalvarro et al [15] propose demand-based coscheduling (DCS) which uses incoming messages to schedule the processes for which they are intended.…”

Section: Communication-driven Coscheduling (Cdc)mentioning

confidence: 99%

“…The simplest approach to time-sharing a cluster is to leave each node to schedule its own processes autonomously. However, this form of scheduling can be very inefficient for parallel jobs that need process synchronization, mainly due to the lack of coordination among local schedulers [11].…”

Section: Introductionmentioning

confidence: 99%

“…Two main strategies for coordinating individual local schedulers have been proposed: gang scheduling (GS) [8], [14] and communication-driven coscheduling (CDC) [2]- [5], [11], [12], [15], [19]. GS uses explicit global synchronization to schedule all the processes of a job simultaneously.…”

Section: Introductionmentioning

confidence: 99%

“…Thus, it will be of benefit to schedule or to keep scheduled the receiver process to reduce synchronization delay. Compared to GS, CDC schemes have better scalability and reliability, and have been shown to be efficient in clusters [4], [11]. Though these advantages, however, one major problem found in the existing CDC schemes is that they do not incorporate any steps to attempt to properly handle priority boost conflicts, leading to degraded performance.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Design and Implementation of a Contention-Aware Coscheduling Strategy on Multi-Programmed Heterogeneous Clusters

Byun

2011

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYCoscheduling has been gained a resurgence of interest as an effective technique to enhance the performance of parallel applications in multi-programmed clusters. However, existing coscheduling schemes do not adequately handle priority boost conflicts, leading to significantly degraded performance. To address this problem, in our previous study, we devised a novel algorithm that reorders the scheduling sequence of conflicting processes based on the rescheduling latency of their correspondents in remote nodes. In this paper, we exhaustively explore the design issues and implementation details of our contention-aware coscheduling scheme over Myrinet-based cluster system. We also practically analyze the impact of various system parameters and job characteristics on the performance of all considered schemes on a heterogeneous Linux cluster using a generic coscheduling framework. The results show that our approach outperforms existing schemes (by up to 36.6% in avg. job response time), reducing both boost conflict ratio and overall message delay.

show abstract

Generalized parallel-server fork-join queues with dynamic task scheduling

et al. 2008

Self Cite

View full text Add to dashboard Cite

This paper introduces a generalization of the classical parallel-server fork-join queueing system in which arriving customers fork into multiple tasks, every task is uniquely assigned to one of the set of single-server queues, and each task consists of multiple iterations of different stages of execution, including task vacations and communication among sibling tasks. Several classes of dynamic polices are considered for scheduling multiple tasks at each of the single-server queues to maintain effective server utilization. The paper presents an exact matrix-analytic analysis of generalized parallel-server fork-join queueing systems, for small instances of the stochastic model, and presents an approximate matrixanalytic analysis and fixed-point solution, for larger instances of the model.

show abstract

Alternatives to Coscheduling a Network of Workstations

Cited by 26 publications

References 20 publications

Dynamic Fractional Resource Scheduling for cluster platforms

Dynamic Fractional Resource Scheduling for cluster platforms

Design and Implementation of a Contention-Aware Coscheduling Strategy on Multi-Programmed Heterogeneous Clusters

Generalized parallel-server fork-join queues with dynamic task scheduling

Contact Info

Product

Resources

About