Hybrid Performance-Oriented Scheduling of Moldable Jobs with QoS Demands in Multiclusters and Grids

Abstract

Abstract. This paper addresses the dynamic scheduling of moldable jobs with QoS demands (soft-deadlines) in multiclusters. A moldable job can be run on a variable number of resources. Three metrics (over-deadline, makespan and idletime) are combined with weights to evaluate the scheduling performance. Two levels of performance optimisation are applied in the multicluster. At the multicluster level, a scheduler (which we call MUSCLE) allocates parallel jobs with high packing potential to the same cluster; MUSCLE also takes the jobs' QoS requirements into account and employs a heuristic to achieve performance balancing across the multicluster. At the single cluster level, an existing workload manager, called TITAN, utilizes a genetic algorithm to further improve the scheduling performance of the jobs allocated by MUSCLE. Extensive experimental studies are conducted to verify the effectiveness of the scheduling mechanism in MUSCLE. The results show that the comprehensive scheduling performance of parallel jobs is significantly improved across the multicluster.

Real-Time Divisible Load Scheduling with Different Processor Available Times

“…However, most of these studies assume the existence of some form of task graph to describe communication and precedence relations between computational units called subtasks (i.e., nodes in the task graph). Despite the increasing importance of arbitrarily divisible applications [29], to the best of our knowledge, only a few researchers [18,20,22] have investigated the real-time scheduling of arbitrarily divisible loads.…”

“…The most closely related work to ours is the scheduling of "scalable tasks" [20] or "moldable jobs" [8,14,18,30,33], where only very few of them [18,20] have considered QoS support. In [22] we investigated real-time cluster-based divisible load scheduling and proposed several algorithms for homogenous clusters.…”

Real-Time Divisible Load Scheduling with Advance Reservation

“…// Reservation admission control 3: if !ResvAdmTest(T) then 4: return false 5: end if 6 TempResvList.add(T) 13: else 14: TempTaskList.add(T) 15: end if 16: // EDF scheduling of regular tasks 17: order TempTaskList by task absolute deadline 18: order TempResvList by reservation start time 19: while TempTaskList != φ do 20: TempTaskList.remove(T) 21: // Regular task admission control 22: if !AdmTest(T) then 23: return false 24: else 25: TempScheduleQueue.add(T) 26: end if 27: end while 28: TaskWaitingQueue ← TempScheduleQueue 29: ResvQueue ← TempResvList 30: return true ing queue of regular tasks. We adopt the EDF (Earliest Deadline First) scheduling algorithm and order the queue by task absolute deadlines.…”

“…As a result, the real-time scheduling of arbitrarily divisible loads is becoming a significant problem for clusterbased research computing facilities like the U.S. CMS Tier-2 sites [33]. Due to the increasing importance [29], a few efforts [16,20,22] have been made in real-time divisible load scheduling, with significant initial progress in important theories and novel approaches.…”

“…However, they either do not consider arbitrarily divisible loads or have no support for advance reservations. Due to the increasing importance of arbitrarily divisible applications [29], a few researchers [16,20,22] have investigated the real-time divisible load scheduling. In our previous work [22,23,21], we applied divisible load theory [34] and proposed several scheduling algorithms for real-time divisible loads.…”