Workload Alleviation Scheduling Framework to Alleviate Negative Performance Impact of Intermediate Data Skew in Small-Scale MapReduce Cloud

Huang, Tzu‐Chi; Chu, K.R.; Lin, Jia-Huei; Huang, Guo-Hao; Shieh, Ce-Kuen

doi:10.1109/icsse.2018.8520003

Cited by 2 publications

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Huang et al [29] proposed a Workload Alleviation Scheduling Framework (WASF) in order to avoid negative effect of intermediate data skew in small scale over MapReduce cloud. The intermediate data skew is caused due to unevenly allocation of intermediate data between nodes at run time.…”

Section: B Resource Allocation Schedulingmentioning

confidence: 99%

Straggler Mitigation in Hadoop MapReduce Framework: A Review

Ajibade¹,

Bakar²,

Aliyu³

et al. 2022

IJACSA

View full text Add to dashboard Cite

Processing huge and complex data to obtain useful information is challenging, even though several big data processing frameworks have been proposed and further enhanced. One of the prominent big data processing frameworks is MapReduce. The main concept of MapReduce framework relies on distributed and parallel processing. However, MapReduce framework is facing serious performance degradations due to the slow execution of certain tasks type called stragglers. Failing to handle stragglers causes delay and affects the overall job execution time. Meanwhile, several straggler reduction techniques have been proposed to improve the MapReduce performance. This study provides a comprehensive and qualitative review of the different existing straggler mitigation solutions. In addition, a taxonomy of the available straggler mitigation solutions is presented. Critical research issues and future research directions are identified and discussed to guide researchers and scholars.

show abstract

Section: B Resource Allocation Schedulingmentioning

confidence: 99%

Straggler Mitigation in Hadoop MapReduce Framework: A Review

Ajibade¹,

Bakar²,

Aliyu³

et al. 2022

IJACSA

View full text Add to dashboard Cite

show abstract

Performance Challenges and Solutions in Big Data Platform Hadoop

Singh

Verma

Madaan

2023

RACSC

View full text Add to dashboard Cite

Background: The present era demands continuous support to bring improvements in executing complex analytics on large-scale data and to work beyond traditional systems. Objective: The need for processing diverse data types and solutions for different domains of the industry is rising. Such needs increase the requirement for sophisticated techniques and methods to enhance the existing platforms and mechanisms further. It provides an opportunity for the research community to investigate further into the existing systems, find potential issues, and propose new ways to improve the current systems. Hadoop is a popular choice to manage and process Big data. It is an open-source platform and a front-runner in the batch processing of large-scale jobs. The economy associated with the cluster in scaling is low as compared to other platforms. However, this popularity by no means guarantees high performance in all scenarios. With the continuous evolution in data development and industrial requirements, it is imperative to investigate and look into new methods and techniques to bring advancements to the existing system. Method: A systematic review is represented in this paper to have an insight into the current progress in this field. Research publications from various sources are taken and analyzed. The performance of a cluster largely depends upon the different job processing mechanisms and policies associated with it. Conclusion: While extensive studies and solutions are proposed, the performance bottlenecks in terms of load balancing, resource utilization, content management, and efficient processing prevail. Not many of the solutions are there on scheduling about the trade-off between different parameters, the process of content splitting and merging is not explored to a large extent and the skew mitigation solutions are more focused on Reduce side of the MapReduce while the Map side is not utilized much for load balancing.

show abstract

Workload Alleviation Scheduling Framework to Alleviate Negative Performance Impact of Intermediate Data Skew in Small-Scale MapReduce Cloud

Cited by 2 publications

References 8 publications

Straggler Mitigation in Hadoop MapReduce Framework: A Review

Straggler Mitigation in Hadoop MapReduce Framework: A Review

Performance Challenges and Solutions in Big Data Platform Hadoop

Contact Info

Product

Resources

About