A Study of Factors Affecting MapReduce Scheduling

Gaur, Manisha; Minocha, Bhawna; Muttoo, Sunil Kumar

doi:10.1007/978-981-10-6620-7_27

Advances in Intelligent Systems and Computing

2017

DOI: 10.1007/978-981-10-6620-7_27

|View full text |Cite

A Study of Factors Affecting MapReduce Scheduling

Manisha Gaur

Bhawna Minocha

Sunil Kumar Muttoo

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2021

Publication Types

Select...

Article1

Relationship

Self Cite0

Independent1

Authors

Journals

Cited by 1 publication

(1 citation statement)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, in a heterogeneous Hadoop cluster, an appropriate task scheduling algorithm is significant to improve the overall performance of the cluster. 13 We propose a spider monkey optimization-based scheduling algorithm (SMOSA). By analyzing the actual computing performance of each node in the cluster, we find the optimal scheduling plan to improve resource utilization and task execution speed.…”

mentioning

confidence: 99%

SMOSA: Spider monkey optimization‐based scheduling algorithm for heterogeneous Hadoop

Zhang

Guan

et al. 2021

Concurrency and Computation

View full text Add to dashboard Cite

Hadoop is a typical framework for processing big data. Task scheduling algorithms have a significant impact on the processing performance of Hadoop clusters. Existing scheduling algorithms of Hadoop fail to consider the performance differences between nodes in heterogeneous Hadoop clusters, causing problems such as uneven task allocation and low resource utilization. Aiming to solve this problem, we propose a spider monkey optimization-based scheduling algorithm (SMOSA) for heterogeneous Hadoop. First, the cluster heartbeat mechanism is used to obtain information such as memories and CPUs of nodes to comprehensively consider the actual load capacity of each node. Then, the spider monkey optimization algorithm is adopted to find the optimal mapping relationship between tasks and resources by taking the task completion time as the objective function and updating the position of the spider monkey.Finally, we calculate the remaining rate of node hardware resources, and according to the task type, the node with the higher remaining rate of resource will give priority to the task. Data are compressed for I/O type tasks to reduce disk operations and increase the speed of task execution. Experimental results show that, compared with existing scheduling algorithms, the SMOSA can effectively reduce task execution time and can significantly improve scheduling efficiency and task execution speed especially in heterogeneous Hadoop clusters. For different types of tasks, the execution time can be reduced by up to 19%.

show abstract

mentioning

confidence: 99%

SMOSA: Spider monkey optimization‐based scheduling algorithm for heterogeneous Hadoop

Zhang

Guan

et al. 2021

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

hi@scite.ai

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

A Study of Factors Affecting MapReduce Scheduling

Cited by 1 publication

References 22 publications

SMOSA: Spider monkey optimization‐based scheduling algorithm for heterogeneous Hadoop

SMOSA: Spider monkey optimization‐based scheduling algorithm for heterogeneous Hadoop

Contact Info

Product

Resources

About