The scheduling techniques in the Hadoop and Spark of smart cities environment: a systematic review

Abstract

Processing extensive and diverse data in real-time is a significant challenge in the context of smart cities. Timely access to information and efficient analytics is essential for smart city services to make data-driven decisions and enhance urban living. Scheduling algorithms play a crucial role in ensuring the prompt delivery of services and efficient task completion. This paper explores various scheduling techniques, including static, dynamic, and hybrid schedulers, and compares their objectives and performance. Additionally, the study examines two prominent data processing frameworks, Hadoop and Spark, and compares their capabilities in handling big data in smart cities. With its ability to process large amounts of data quickly and efficiently, Spark has shown superiority over Hadoop in real-time data processing and performance optimization. The paper concludes by highlighting the strengths and limitations of each framework. It discusses the need for further research in optimizing scheduling techniques and exploring hybrid artificial intelligence scheduling for Spark. Overall, the findings contribute to a better understanding of data processing in real-time and provide insights for researchers and practitioners in smart cities.