Optimized Capacity Scheduler for MapReduce Applications in Cloud Environments

Lakshmi, Adepu Sree; Chandra, N. Subhash; BalRaju, M.

doi:10.1007/978-981-13-1402-5_12

Cited by 3 publications

(1 citation statement)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The scheduler uses the concept of queues. Each queue is allocated to an organisation or user, and resources are divided among these queues [13], [14]. Jobs are distributed in multiple queues according to their conditions and allocate specific capacity for each queue.…”

Section: B Hadoop Schedulermentioning

confidence: 99%

Asynchronous Non-Blocking Algorithm to Handle Straggler Reduce Tasks in Hadoop System

Khoiruddin¹,

Zakaria²,

Alhussian³

2020

Int. J. Adv. Sci. Eng. Inf. Technol.

View full text Add to dashboard Cite

Hadoop is widely adopted as a big data processing application as it can run on commercial hardware at a reasonable time. Hadoop uses asynchronous blocking concurrency using Thread and Future class. Therefore, in some cases such as network link or hardware failure, a running task may block other tasks from running (the task becomes straggler). Hadoop releases are equipped with algorithms to handle straggler tasks problem. However, the algorithms manage Map and Reduce task similarly, while the straggler root cause might be different for both tasks. In this paper, the Asynchronous Non-Blocking (ANB) method is proposed to improve the performance and avoid the blocking of Reduce task in Hadoop. Instead of using the single queue, our approach uses two queues, i.e. task queue and callback queue. When a task is not ready or detected as a straggler, it is removed from the main task queue and temporarily sent to the callback queue. When the task is ready to run, it will be sent back to the main task queue for running. The performance of the algorithm is compared with rTuner, the latest paper found on handling straggler task in Reduce task. From the comparison, it is shown that ANB consistently gives faster time to complete because any unready tasks will be directly put into the callback queue without blocking other tasks. Furthermore, the overhead time in rTuner is high as it needs to check the straggler status and to find the reason for a task to become straggler.

show abstract

Section: B Hadoop Schedulermentioning

confidence: 99%

Asynchronous Non-Blocking Algorithm to Handle Straggler Reduce Tasks in Hadoop System

Khoiruddin¹,

Zakaria²,

Alhussian³

2020

Int. J. Adv. Sci. Eng. Inf. Technol.

View full text Add to dashboard Cite

show abstract

Tools for Fault and Reliability in Multilayered Cloud

Gupta

2020

Trust &Amp; Fault in Multi Layered Cloud Computing Architecture

View full text Add to dashboard Cite

Review on Task Scheduling in Ubiquitous Clouds

Kumar

2019

JISMAC

View full text Add to dashboard Cite

The cloud being a prominent area for research, is ubiquitous as it serves the client needs irrespective of time and place. It has become the most preferred network due to its guaranteed service rendering and cost effectiveness. The increase in the capability of the cloud services has increased the number of users to adapt to cloud. The increase in the adaption towards cloud in turn results with insufficiency in the proper and the simultaneous allocation of the resources according to the requisitions. So the task scheduling and resource allocation for the cloud becomes essential. As the conventional methods of task scheduling arrive to a local optima solution that are less-effective, the paper surveys the meta-heuristic optimization based task scheduling and the resource allocation for the ubiquitous cloud environment, that arrives to an more optimal solution at a faster rate and at ease. The paper presents the survey of the optimization techniques of task scheduling available for the cloud and the discussion of the improvement of the performance metrics in terms of make-span, throughput, cost, latency and successful service provisioning compared to the other conventional methods.

show abstract

Optimized Capacity Scheduler for MapReduce Applications in Cloud Environments

Cited by 3 publications

References 13 publications

Asynchronous Non-Blocking Algorithm to Handle Straggler Reduce Tasks in Hadoop System

Asynchronous Non-Blocking Algorithm to Handle Straggler Reduce Tasks in Hadoop System

Tools for Fault and Reliability in Multilayered Cloud

Review on Task Scheduling in Ubiquitous Clouds

Contact Info

Product

Resources

About