A Constraint Programming-Based Resource Management Technique for Processing MapReduce Jobs with SLAs on Clouds

Lim, Norman; Majumdar, Shikharesh; Ashwood-Smith, Peter

doi:10.1109/icpp.2014.50

Cited by 15 publications

(25 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This section discusses the results of the experiments conducted to compare the performance of RM-DCWF with that of MRCP-RM [29] (described in Section "Related Work"). Note that MRCP-RM is only applicable to jobs with two phases of execution, such as MapReduce [7] jobs, whereas in addition to MapReduce jobs, RM-DCWF can also handle jobs with different structures and more than two execution phases.…”

Section: Comparison With Mcrp-rmmentioning

confidence: 99%

“…A walkthrough of Table 1 is provided next. Note that the distributions used to generate the parameters of the workload, including the job arrival rate, earliest start time of jobs, and job deadlines are adopted from [11,29]. The first component of the table describes the workload.…”

Section: System and Workload Parameters For The Factor-at-a-time Expementioning

confidence: 99%

“…Note that MRCP-RM is only applicable to jobs with two phases of execution, such as MapReduce [7] jobs, whereas in addition to MapReduce jobs, RM-DCWF can also handle jobs with different structures and more than two execution phases. Thus, the workload that is used in this comparison is a synthetic MapReduce workload that is used and described in [29]. RM-DCWF is configured to use PD-SL-TSP1, which is observed to have the best performance when processing this MapReduce workload.…”

Section: Comparison With Mcrp-rmmentioning

confidence: 99%

“…In [29], a MapReduce Constraint Programming based Resource Management technique (MRCP-RM) is presented for processing an open stream of MapReduce jobs with SLAs, where each SLA comprises an earliest start time, an execution time, and a deadline. The objective of MRCP-RM is to minimize the number of jobs that miss their deadlines.…”

Section: Resource Management Techniques On Distributed Systems For Prmentioning

confidence: 99%

“…The synthetic workloads used in the experiments are based on real scientific workflows described in the literature. A comparison of the performance of the proposed technique with that of a conventional first-in-firstout (FIFO) scheduler as well as a technique from the literature [29] that has objectives similar to RM-DCWF is presented.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A resource management technique for processing deadline-constrained multi-stage workflows

2017

Self Cite

View full text Add to dashboard Cite

The use of cloud computing that provides resources on demand to various types of users, including enterprises as well as engineering and scientific institutions, is growing rapidly. An effective resource management middleware is necessary to harness the power of the underlying distributed hardware in a cloud. Two of the key operations provided by a resource manager are resource allocation (matchmaking) and scheduling. This paper concerns the problem of matchmaking and scheduling an open stream of multi-stage jobs (or workflows) with Service Level Agreements (SLAs) on a cloud or cluster. Multi-stage jobs require service from multiple system resources and are characterized by multiple phases of execution. This paper presents a resource allocation and scheduling technique called RM-DCWF: Resource Management Technique for Deadline-constrained Workflows that can efficiently matchmake and schedule an open stream of multi-stage jobs with SLAs, where each SLA is characterized by an earliest start time, an execution time, and a deadline. A rigorous simulation-based performance evaluation of RM-DCWF is conducted using synthetic workloads derived from real scientific workflows. In addition, the impact of various system and workload parameters on system performance is investigated. The results of this performance evaluation demonstrate the effectiveness of RM-DCWF as captured in a low number of jobs missing their deadlines.

show abstract

Section: Comparison With Mcrp-rmmentioning

confidence: 99%

Section: System and Workload Parameters For The Factor-at-a-time Expementioning

confidence: 99%

Section: Comparison With Mcrp-rmmentioning

confidence: 99%

Section: Resource Management Techniques On Distributed Systems For Prmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A resource management technique for processing deadline-constrained multi-stage workflows

2017

Self Cite

View full text Add to dashboard Cite

show abstract

Budget‐constraint stochastic task scheduling on heterogeneous cloud systems

Tang

2017

Concurrency and Computation

View full text Add to dashboard Cite

Summary In the past few years, more and more business‐to‐consumer and enterprise applications run in the heterogeneous clouds. Such cloud bag‐of‐tasks applications are usually budget constrained, and their scheduling is an essential problem for cloud provider. The problem is even more complex and challenging when the accurate knowledge about task execution time is unknown in advance. Focusing on these challenges, we first build a cloud resource management architecture and stochastic task model, which divides cloud task into two execution parts. Then, we deduce bag‐of‐tasks applications' schedule length (Makespan) and total cost according to heterogeneous clouds' online feedback information of task first part execution. Thirdly, we formulate this stochastic scheduling problem as a linear programming problem. Lastly, we propose a time and cost multiobjective stochastic task scheduling genetic algorithm, in which can find Pareto optimal schedules for stochastic cloud task that meet its budget constraint. The extensive simulation experiments were carried out on a heterogeneous cloud platform with 400 virtual machines, and tasks were derived from Parallel Workloads Archive and the analysis data of real‐world cloud systems. The experimental results show that our proposed stochastic task scheduling genetic algorithm can get shorter schedule length and lower cost with task budget constraints.

show abstract

On Singleton Arc Consistency for CSPs Defined by Monotone Patterns

et al. 2018

View full text Add to dashboard Cite

Singleton arc consistency is an important type of local consistency which has been recently shown to solve all constraint satisfaction problems (CSPs) over constraint languages of bounded width. We aim to characterise all classes of CSPs defined by a forbidden pattern that are solved by singleton arc consistency and closed under removing constraints. We identify five new patterns whose absence ensures solvability by singleton arc consistency, four of which are provably maximal and three of which generalise 2-SAT. Combined with simple counter-examples for other patterns, we make significant progress towards a complete classification.

show abstract

A Constraint Programming-Based Resource Management Technique for Processing MapReduce Jobs with SLAs on Clouds

Cited by 15 publications

References 10 publications

A resource management technique for processing deadline-constrained multi-stage workflows

A resource management technique for processing deadline-constrained multi-stage workflows

Budget‐constraint stochastic task scheduling on heterogeneous cloud systems

On Singleton Arc Consistency for CSPs Defined by Monotone Patterns

Contact Info

Product

Resources

About