Hybrid Metaheuristic Based Offline Parallel Job Scheduling in Heterogeneous Computing Systems

Abstract

Over two decades, Heterogeneous Computing Systems (HCS) are offering large amount of federated computing resources, spanning across different administrative domains, to compute-intensive user applications. Efficient job schedulers are required to allocate HCS resources to user applications to satisfy system provider and user requirements. Offline scheduling is most popular kind of job scheduling in heterogeneous system, in which jobs are collected in batch and scheduled together. Job scheduling in HCS has become NP-hard problem due to system scale, federated structure and high resource as well as job heterogeneity. Simple queuing and deterministic heuristics have failed to provide optimal solution to NP-hard job scheduling problem. Due to NP-hard nature of job scheduling problem, there is always a scope to propose new scheduling solutions using meta-heuristics. Offline scheduling in HCS has been focused more on scheduling independent sequential tasks viz. Bag-of-tasks or Many-tasks. Offline scheduling of parallel jobs (composed of collaborating tasks with no precedence) in HCS has not gained much attention. In this paper, a novel hybrid multi-objective meta-heuristic known as HCSPSO, which combines the qualities of Cuckoo search (CS) and Particle Swarm Optimization (PSO), has been proposed to schedule batch of parallel jobs in multi-cluster HCS platform. Proposed HCSPSO policy is extensively compared with different heuristics and metaheuristics using different resource configurations and real supercomputing workload logs. Comparative results have showed the dominance of the proposed hybrid scheduling algorithm over other algorithms.