Multiple job co-allocation strategy for heterogeneous multi-cluster systems based on linear programming

Blanco, Héctor; Lérida, Josep L.; Cores, Fernando; Guirado, Fernando

doi:10.1007/s11227-011-0596-2

Cited by 12 publications

(6 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Authors in [19] concluded that offline scheduling by treating a group of parallel jobs together for scheduling produced better results than allocating each job one by one from waiting queue. Authors in [20] used Mixed-Integer programming (MIP) model, to propose a job execution ordering technique based to decide order of waiting jobs using sets of job packages. Proposed solution took longer time and not feasible for scheduling bigger problems in large scale systems.…”

Section: A Scheduling On Parallel Jobsmentioning

confidence: 99%

“…Most of the research works listed in this section focused on scheduling either independent jobs or parallel jobs consisting of independent tasks in HCS platforms without considering bandwidth penalties. We found very few research works [15][16], [18][19][20] which provide offline solution for scheduling collaborative parallel jobs by considering computation heterogeneity and inter-cluster communications. Such parallel job's tasks have computation and communication phases and tasks can collaborative with each other during communication phase.…”

Section: B Scheduling On Independent Sequential Jobsmentioning

confidence: 99%

“…The Heterogeneous Computing System model and offline scheduling problem are defined as follows (Blanco 2011). Heterogeneous computing system consists of multiple federated clusters connected by a dedicated communication link.…”

Section: A Heterogeneous Computing System and Parallel Batch Jobsmentioning

confidence: 99%

See 2 more Smart Citations

Hybrid Metaheuristic Based Offline Parallel Job Scheduling in Heterogeneous Computing Systems

Chhabra*,

Singh,

Kahlon

2019

IJRTE

View full text Add to dashboard Cite

Over two decades, Heterogeneous Computing Systems (HCS) are offering large amount of federated computing resources, spanning across different administrative domains, to compute-intensive user applications. Efficient job schedulers are required to allocate HCS resources to user applications to satisfy system provider and user requirements. Offline scheduling is most popular kind of job scheduling in heterogeneous system, in which jobs are collected in batch and scheduled together. Job scheduling in HCS has become NP-hard problem due to system scale, federated structure and high resource as well as job heterogeneity. Simple queuing and deterministic heuristics have failed to provide optimal solution to NP-hard job scheduling problem. Due to NP-hard nature of job scheduling problem, there is always a scope to propose new scheduling solutions using meta-heuristics. Offline scheduling in HCS has been focused more on scheduling independent sequential tasks viz. Bag-of-tasks or Many-tasks. Offline scheduling of parallel jobs (composed of collaborating tasks with no precedence) in HCS has not gained much attention. In this paper, a novel hybrid multi-objective meta-heuristic known as HCSPSO, which combines the qualities of Cuckoo search (CS) and Particle Swarm Optimization (PSO), has been proposed to schedule batch of parallel jobs in multi-cluster HCS platform. Proposed HCSPSO policy is extensively compared with different heuristics and metaheuristics using different resource configurations and real supercomputing workload logs. Comparative results have showed the dominance of the proposed hybrid scheduling algorithm over other algorithms.

show abstract

Section: A Scheduling On Parallel Jobsmentioning

confidence: 99%

Section: B Scheduling On Independent Sequential Jobsmentioning

confidence: 99%

See 1 more Smart Citation

Hybrid Metaheuristic Based Offline Parallel Job Scheduling in Heterogeneous Computing Systems

Chhabra*,

Singh,

Kahlon

2019

IJRTE

View full text Add to dashboard Cite

show abstract

“…These techniques, however, are based on the job arrival order, only moving jobs forward that accomplish specific deadline requirements. Blanco et al [14,15] proposed diverse techniques for determining the best scheduling of sets of job packages, proposing a new job execution order to minimize their overall execution time, based on a MixedInteger programming model. Due to the intractable nature of the problem, it is desirable to explore other avenues for developing good heuristic algorithms for the problem.…”

Section: Related Workmentioning

confidence: 99%

Blacklist muti-objective genetic algorithm for energy saving in heterogeneous environments

et al. 2016

Self Cite

View full text Add to dashboard Cite

Reducing energy consumption in large-scale computing facilities has become a major concern in recent years. Most of the techniques have focused on determining the computing requirements based on load predictions and thus turning unnecessary nodes on and off. Nevertheless, once the available resources have been configured, new opportunities arise for reducing energy consumption by providing optimal matching of parallel applications to the available computing nodes. Current research in scheduling has concentrated on not only optimizing the energy consumed by the processors but also optimizing the makespan, i.e., job completion time. The large number of heterogeneous computing nodes and variability of application-tasks are factors that make the scheduling an NP-Hard problem. Our aim in this paper is a multi-objective genetic algorithm based on a weighted blacklist able to generate scheduling decisions that globally optimizes the energy consumption and the makespan.

show abstract

“…These techniques however are based on predetermined order, only moving forward jobs that accomplish specific deadline requirements. Blanco et al (2011Blanco et al ( , 2012 proposed diverse techniques for determining the best scheduling of sets of job packages to minimize their overall execution time, based on a Mixed-Integer programming model. Although these techniques produce very good results, their computational cost makes them impractical for large-scale environments.…”

Section: Related Workmentioning

confidence: 99%

Multi-criteria genetic algorithm applied to scheduling in multi-cluster environments

Gabaldon

Lérida

Guirado

et al. 2015

Journal of Simulation

View full text Add to dashboard Cite

Scheduling and resource allocation to optimize performance criteria in multi-cluster heterogeneous environments is known as an NP-hard problem, not only for the resource heterogeneity, but also for the possibility of applying co-allocation to take advantage of idle resources across clusters. A common practice is to use basic heuristics to attempt to optimize some performance criteria by treating the jobs in the waiting queue individually. More recent works proposed new optimization strategies based on Linear Programming techniques dealing with the scheduling of multiple jobs simultaneously. However, the time cost of these techniques makes them impractical for large-scale environments. Population-based meta-heuristics have proved their effectiveness for finding the optimal schedules in large-scale distributed environments with high resource diversification and large numbers of jobs in the batches. The algorithm proposed in the present work packages the jobs in the batch to obtain better optimization opportunities. It includes a multi-objective function to optimize not only the Makespan of the batches but also the Flowtime, thus ensuring a certain level of QoS from the users' point of view. The algorithm also incorporates heterogeneity and bandwidth awareness issues, and is useful for scheduling jobs in large-scale heterogeneous environments. The proposed meta-heuristic was evaluated with a real workload trace. The results show the effectiveness of the proposed method, providing solutions that improve the performance with respect to other well-known techniques in the literature.

show abstract

Multiple job co-allocation strategy for heterogeneous multi-cluster systems based on linear programming

Cited by 12 publications

References 10 publications

Hybrid Metaheuristic Based Offline Parallel Job Scheduling in Heterogeneous Computing Systems

Hybrid Metaheuristic Based Offline Parallel Job Scheduling in Heterogeneous Computing Systems

Blacklist muti-objective genetic algorithm for energy saving in heterogeneous environments

Multi-criteria genetic algorithm applied to scheduling in multi-cluster environments

Contact Info

Product

Resources

About