Scheduling Concurrent Workflows in HPC Cloud through Exploiting Schedule Gaps

Jiang, He-Jhan; Huang, Kuo-Chan; Chang, Hsi-Ya; Gu, Di-Syuan; Shih, Po‐Jen

doi:10.1007/978-3-642-24650-0_24

Cited by 26 publications

(21 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This, if happening, would degrade the overall system performance. Jiang et al proposed a distributed gap search scheme to remedy such situations in [6]. It is our future work to incorporate the enhanced best-fit approach into the distributed gap search scheme to further improve multiple workflow scheduling performance.…”

Section: B Experimental Resultsmentioning

confidence: 95%

Scheduling Multiple Scientific and Engineering Workflows through Task Clustering and Best-Fit Allocation

Tsai

Huang

Chang

et al. 2012

2012 IEEE Eighth World Congress on Services

Self Cite

View full text Add to dashboard Cite

Most previous workflow scheduling research focused on scheduling a single workflow on parallel systems. Recent researches show that utilizing idle time slots between scheduled tasks is a promising direction for efficient multiple workflow scheduling. Stavrinides and Karatza proposed a list scheduling approach to efficient utilization of the idle time slots through bin packing techniques. In this paper, we elaborate on this direction and develop a new approach to further improve multiple workflow scheduling performance through two techniques. The first, in contrast with the list scheduling approach, is clustering the tasks within workflows into groups before allocation. This can reduce inter-task communication cost and thus improve workflow execution performance. The second technique tries to make a balance between tasks' start time and the fitness of idle time slots when allocating task groups. The proposed approach has been evaluated with a series of simulation experiments and compared to the previous method. The results show that our approach outperforms the previous method significantly, up to 51% performance improvement in terms of average makespan.

show abstract

Section: B Experimental Resultsmentioning

confidence: 95%

Scheduling Multiple Scientific and Engineering Workflows through Task Clustering and Best-Fit Allocation

Tsai

Huang

Chang

et al. 2012

2012 IEEE Eighth World Congress on Services

Self Cite

View full text Add to dashboard Cite

show abstract

“…Due to the NP-completeness of the problem, a particle swarm optimization algorithm is presented to find near optimal solutions. Jiang et al [15] consider the problem of concurrent workflow scheduling in high performance computing resources (HPC clouds). They present a scheduling method that aims to minimize the total cost in terms of the computation cost, the communication cost as well as the earliest start time.…”

Section: Related Workmentioning

confidence: 99%

“…An important strong assumption that theoretical scheduling models typically impose is that task resource demand can be represented by a piecewise constant function (i.e. boxes) [15] [16]. Importantly, boxes are unable to capture the dynamicity of resource utilization patterns inherit within Cloud computing [17].…”

Section: Introductionmentioning

confidence: 99%

Resource boxing

Primas

Garraghan

Djemame

et al. 2016

Proceedings of the 9th International Conference on Utility and Cloud Computing

View full text Add to dashboard Cite

Scheduling is a core component within distributed systems to determine optimal allocation of tasks within servers. This is challenging within modern Cloud computing systems comprising millions of tasks executing in thousands of heterogeneous servers. Theoretical scheduling is capable of providing complete and sophisticated algorithms towards a single objective function. However, Cloud computing systems pursue multiple and oftentimes conflicting objectives towards provisioning high levels of performance, availability, reliability and energy-efficiency. As a result, theoretical scheduling for Cloud computing is performed by simplifying assumptions for applicability. This is especially true for task utilization patterns, which fluctuate in practice yet are modelled as piecewise constant in theoretical scheduling models. While there exists work for modelling dynamic Cloud task patterns for evaluating applied scheduling, such models are incompatible with the inputs needed for theoretical scheduling which require such patterns to be represented as boxes. Presently there exist no methods capable of accurately converting real task patterns derived from empirical data into boxes. This results in a significant gap towards theoreticians understanding and proposing algorithms derived from realistic assumptions towards enhanced Cloud scheduling. This work proposes resource boxing an approach for automated conversion of realistic task patterns in Cloud computing directly into box inputs for theoretical scheduling. We propose numerous resource conversion algorithms capable of accurately representing real task utilization patterns in the form of scheduling boxes. Algorithms were evaluated using production Cloud trace data, demonstrating a difference between real utilization and scheduling boxes less than 5%. We also provide an application for how resource boxing can be exploited to directly translate research from the applied community into the theoretical community.

show abstract

“…Furthermore, Jiang et al [19] present a method which exploits gaps formed when scheduling concurrent workflows in HPC cloud. At the first step, a PCH (Path Clustering Heuristic) algorithm is applied to cluster the tasks within a workflow into different task groups.…”

Section: ) Scheduling Multiple Workflowsmentioning

confidence: 99%

Workflow Scheduling in Cloud Computing: A Survey

Fakhfakh

Kacem

2014

2014 IEEE 18th International Enterprise Distributed Object Computing Conference Workshops and Demonstrations

View full text Add to dashboard Cite

Workflow systems have become a major vehicle for easy and efficient development of scientific applications. This type of systems can benefit from the resource provisioning technology offered by the cloud computing. In fact, the latter offers on-demand virtualized resources to its users. These virtual resources can be added and released dynamically. Also, users are charged on a pay-per-use basis. How to make appropriate decisions when allocating resources to the tasks and dispatching the computing tasks to resource pool has become the main issue in cloud computing. The amount of allocated resources affects the execution time of the applications and the cost incurred by the user. In fact, resource under-provisioning will necessarily affect the performance. In contrast, over-provisioning can result in idle instances and cause additional costs. Then, efficient scheduling algorithms are required for selection of best suitable resources for task execution. This paper focuses on some of the important workflow scheduling strategies. It brings out an exhaustive survey of such strategies in cloud computing and includes a detailed classification of them. Then, it presents a comparative analysis of the studied approaches. Finally, it stands out a critical challenge for further research.

show abstract

Scheduling Concurrent Workflows in HPC Cloud through Exploiting Schedule Gaps

Cited by 26 publications

References 23 publications

Scheduling Multiple Scientific and Engineering Workflows through Task Clustering and Best-Fit Allocation

Scheduling Multiple Scientific and Engineering Workflows through Task Clustering and Best-Fit Allocation

Resource boxing

Workflow Scheduling in Cloud Computing: A Survey

Contact Info

Product

Resources

About