Scheduling Multiple Scientific and Engineering Workflows through Task Clustering and Best-Fit Allocation

Abstract

Most previous workflow scheduling research focused on scheduling a single workflow on parallel systems. Recent researches show that utilizing idle time slots between scheduled tasks is a promising direction for efficient multiple workflow scheduling. Stavrinides and Karatza proposed a list scheduling approach to efficient utilization of the idle time slots through bin packing techniques. In this paper, we elaborate on this direction and develop a new approach to further improve multiple workflow scheduling performance through two techniques. The first, in contrast with the list scheduling approach, is clustering the tasks within workflows into groups before allocation. This can reduce inter-task communication cost and thus improve workflow execution performance. The second technique tries to make a balance between tasks' start time and the fitness of idle time slots when allocating task groups. The proposed approach has been evaluated with a series of simulation experiments and compared to the previous method. The results show that our approach outperforms the previous method significantly, up to 51% performance improvement in terms of average makespan.

Agent-based load balancing in Cloud data centers

“…A total of four commonly used load balancing heuristics [7,24,40,49] were implement into FAs to determine the initial destination hosts of newly arrived VMs:…”

“…The number of available cores was selected over the amount of free memory because virtualized hosts may assign considerably more (virtual) memory than their available physical memory to VMs [35]. See [40] for an example of a best-fit load balancing heuristic.…”

“…Objective The experiment was designed to evaluate and compare the efficiency of SMAs adopting MapLoad (with and without initial allocation assistance from FAs) and a set of commonly used benchmark heuristics [7,24,40,49] in balancing loads in heterogeneous Cloud data centers.…”

“…4.4). It should be noted that the set of commonly used benchmark heuristics [7,24,40,49] does not make use of live migration of VMs, and thus, once the central manager has allocated a VM to a host, the VM is not migrated. The architecture of the centralized approach consists of a pool of hosts (within a Cloud data center), and a central manager that selects the host to which VM allocation requests should be routed based on a commonly used load balancing heuristic.…”

“…In addition, to explore the efficiency of SMAs adopting MapLoad fully assisted by FAs, FAs and the central manager were provided with the same four commonly used load balancing heuristics. A centralized load balancing approach was selected for comparison because, firstly, centralized approaches are commonly used [7,24,40,49], and secondly, because comparing a centralized load balancing approach with agents adopting MapLoad is a challenge. For instance, whereas centralized approaches (i) have full knowledge of the Cloud data center, e.g., a central manager knows the number of hosts and their latest resource usages [32], and (ii) all the hosts in the Cloud data center are fully controlled by a central manager such as in [32], agents adopting MapLoad (i) have a partial view of the Cloud data center, e.g., an SMA may not be aware of the number of hosts and/or other SMAs' resource usages, and (ii) agents (in this work) should dynamically coordinate among themselves to balance a load across a set of hosts.…”