BAR: An Efficient Data Locality Driven Task Scheduling Algorithm for Cloud Computing

Jin, Jin; Luo, Bin; Song, Xianfang; Dong, Shen; Xiong, Jie

doi:10.1109/ccgrid.2011.55

Cited by 89 publications

(58 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [4] heuristic task scheduling algorithms in Grid considering network communication have been provided. The authors of [5] have presented the scheduling algorithm that accounts for the task input data allocation in the distributed system. In [6] the approach that considers the network and resource status dynamics in task scheduling has been proposed.…”

Section: Literature Review and Problem Statementmentioning

confidence: 99%

The scheduler for the gridsystem based on the parameters monitoring of the computer components

Mukhin

Kornaga

et al. 2017

EEJET

View full text Add to dashboard Cite

Section: Literature Review and Problem Statementmentioning

confidence: 99%

The scheduler for the gridsystem based on the parameters monitoring of the computer components

Mukhin

Kornaga

et al. 2017

EEJET

View full text Add to dashboard Cite

“…In this process, most work focus on finding the appropriate assignment scheme for each job, and the goal of task-level scheduling is to assign tasks to VMs or nodes so that the total execution time of a job can be minimized with the cost limitation. Related work is done by [17][18][19][20][21]. In [18], a directed acyclic graph is used to model the precedence constraints among tasks, in which the task-level scheduling for the sequential, the parallel, and the mixed structures are optimized, respectively.…”

Section: Related Workmentioning

confidence: 99%

Optimal media service scheduling in clouds

You¹,

Qiao

Zhong

et al. 2014

16th International Conference on Advanced Communication Technology

View full text Add to dashboard Cite

Abstract-Cloud becomes increasingly important currently, and media services, which usually have multiple steps and complexity computing requirement, are applied in it. Although this scheduling topic has been investigated a lot, most of works focus on balancing jobs or decreasing the execution time of one job. For large scale paralleling media services, the job request usually contains a series tasks, and how to schedule them to a cloud or a grid with high efficiency and low cost is an important problem. To achieve this target, we focus on the fine grain tasks for media services and propose a task-based scheduling algorithm, in which two-stage scheduling approach is designed, including initializing stage and adjustment stage. In this algorithm, after the initializing assignment process, in order to meet the need of the limited cost consumption, the characters of nodes or virtual machines(VMs), such as balance character, capability character, are considered deeply in scheduling adjusting process. Evaluation shows that our algorithm presents obviously better performance in terms of makespan, cost consumption, and balance factor than several benchmarks. .

show abstract

“…Hadoop's default scheduling techniques (i.e., Capacity [3] and Fair [4] schedulers), typically rely on exploiting data locality in the cluster, i.e., favoring query shipping. Moreover, other, more advanced scheduling proposals, e.g., [9,15], to mention a few, also favor query shipping and exploiting data locality in Hadoop, claiming that it is crutial for performance of MapReduce jobs. The approach in [15] in addition proposes techniques that address the conflict between data locality and fairness in scheduling MapReduce jobs.…”

Section: Related Workmentioning

confidence: 99%

“…Current solutions, including the popular Apache Hadoop [13], provide fault-tolerant, reliable, and scalable platforms for distributed data processing. However, network traffic is identified as a bottleneck for the performance of such systems [9]. Thus, current scheduling techniques typically follow a query shipping approach where the tasks are brought to their input data, hence data locality is exploited for reducing network traffic.…”

Section: Introductionmentioning

confidence: 99%

H-WorD: Supporting Job Scheduling in Hadoop with Workload-Driven Data Redistribution

Jovanovic

Romero

Calders

et al. 2016

Advances in Databases and Information Systems

View full text Add to dashboard Cite

Abstract. Today's distributed data processing systems typically follow a query shipping approach and exploit data locality for reducing network traffic. In such systems the distribution of data over the cluster resources plays a significant role, and when skewed, it can harm the performance of executing applications. In this paper, we address the challenges of automatically adapting the distribution of data in a cluster to the workload imposed by the input applications. We propose a generic algorithm, named H-WorD, which, based on the estimated workload over resources, suggests alternative execution scenarios of tasks, and hence identifies required transfers of input data a priori, for timely bringing data close to the execution. We exemplify our algorithm in the context of MapReduce jobs in a Hadoop ecosystem. Finally, we evaluate our approach and demonstrate the benefits of automatic data redistribution.

show abstract

BAR: An Efficient Data Locality Driven Task Scheduling Algorithm for Cloud Computing

Cited by 89 publications

References 17 publications

The scheduler for the gridsystem based on the parameters monitoring of the computer components

The scheduler for the gridsystem based on the parameters monitoring of the computer components

Optimal media service scheduling in clouds

H-WorD: Supporting Job Scheduling in Hadoop with Workload-Driven Data Redistribution

Contact Info

Product

Resources

About

BAR: An Efficient Data Locality Driven Task Scheduling Algorithm for Cloud Computing

Cited by 89 publications

References 17 publications

The scheduler for the grid­system based on the parameters monitoring of the computer components

The scheduler for the grid­system based on the parameters monitoring of the computer components

Optimal media service scheduling in clouds

H-WorD: Supporting Job Scheduling in Hadoop with Workload-Driven Data Redistribution

Contact Info

Product

Resources

About

The scheduler for the gridsystem based on the parameters monitoring of the computer components

The scheduler for the gridsystem based on the parameters monitoring of the computer components