MapReduce based on serverless platforms

Abstract

This study intends to investigate the application of the MapReduce (MR) framework based on serverless computing in big data processing. By combining the MapReduce model with serverless computing, efficient data processing is achieved. In this framework, the phases of Map task execution, reduce task execution, etc. are accomplished through stateless serverless functions, and data storage is realized with the help of cloud storage platforms (e.g., OSS). In this paper, the author introduces the basic theory of MR, the basic theory of serverless computing, describes the framework implementation process, and discusses the role of OSS in distributed computing. The outcomes of the trial indicate the average execution time of the framework for a WordCount task on 100 pieces of TOEFL English reading data is 6.81 seconds. The discussion analyzes the frameworks advantages (high elasticity, resource utilization) and disadvantages (cold start latency, unsuitable for long time tasks). Future research directions encompass performance optimization, long-time task processing, state management, etc. In summary, this study provides valuable insights for the practical application of serverless computing in big data processing.