Serverless Architecture for Big Data Analytics

Rahman, Mijanur; Hasan, Hasibul

doi:10.1109/gcat47503.2019.8978443

Cited by 11 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It provides faster startup times, lower cost and individual task based scaling [8]. Serverless helps enterprises to focus on application rather than infrastructure [5]. In this paper, we explore usage of serverless infrastructure for adaptive provisioning of batch processing workload to optimize system utilization.…”

Section: Literature Surveymentioning

confidence: 99%

See 1 more Smart Citation

Evaluating Serverless Architecture for Big Data Enterprise Applications

Aimer¹,

Roy²,

Park³

2021

Preprint

View full text Add to dashboard Cite

Migration of enterprise applications to the cloud has been driven by a myriad of benefits ranging from availability of infinite computing resources to the elimination of upfront CapEx cost. However, many users still face the burden of complex framework knowledge requirement to efficiently deploy applications in the cloud. In this paper, we investigate serverless computing for performing large scale data processing with cloudnative primitives. Serverless computing environment abstracts all infrastructure handling, simplifying developers' work who aspire to deploy applications on the cloud. With dynamic input load on the system, serverless architecture has promise to provide better resource utilization and lower costs.

show abstract

Section: Literature Surveymentioning

confidence: 99%

“…The cost model for serverless infrastructure is true pay-per-use, since no costs are involved for the user unless the function is invoked, as opposed to traditional cost models for other cloud offerings, where Virtual Machines are typically billed per-second of uptime regardless of their actual usage [5].…”

Section: Introductionmentioning

confidence: 99%

Evaluating Serverless Architecture for Big Data Enterprise Applications

Aimer¹,

Roy²,

Park³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Traditional deployment methods of MapReduce based on virtual machines or physical machines have limitations in resource utilization, elastic expansion, etc., leading to resource waste and startup delays. Moreover, developers need to pay attention to numerous underlying details, including cluster configuration and scheduling, which undoubtedly increase development and maintenance costs [1]. These limitations constrain the flexibility and efficiency of big data processing.…”

Section: Introductionmentioning

confidence: 99%

MapReduce based on serverless platforms

Zhang

2024

ACE

View full text Add to dashboard Cite

This study intends to investigate the application of the MapReduce (MR) framework based on serverless computing in big data processing. By combining the MapReduce model with serverless computing, efficient data processing is achieved. In this framework, the phases of Map task execution, reduce task execution, etc. are accomplished through stateless serverless functions, and data storage is realized with the help of cloud storage platforms (e.g., OSS). In this paper, the author introduces the basic theory of MR, the basic theory of serverless computing, describes the framework implementation process, and discusses the role of OSS in distributed computing. The outcomes of the trial indicate the average execution time of the framework for a WordCount task on 100 pieces of TOEFL English reading data is 6.81 seconds. The discussion analyzes the frameworks advantages (high elasticity, resource utilization) and disadvantages (cold start latency, unsuitable for long time tasks). Future research directions encompass performance optimization, long-time task processing, state management, etc. In summary, this study provides valuable insights for the practical application of serverless computing in big data processing.

show abstract

“…In Software-as-a-Service (SaaS) model, management responsibility on all layers is abstracted from the user. The serverless paradigm is introduced to provide automated orchestration of deployment and scalable platform that runs based on demand [4]. It is similar to the SaaS model in layer abstraction, but it allows the developer to run any code on the serverless service.…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we implemented a proof-of-concept to explore the applicability of the proposed system on a public cloud provider. In addition, we designed sample workflows to challenge the system with the most used complex and long-running processes in the The serverless paradigm is introduced to provide automated orchestration of deployment and scalable platform that runs based on demand [4]. It is similar to the SaaS model in layer abstraction, but it allows the developer to run any code on the serverless service.…”

Section: Introductionmentioning

confidence: 99%

Serverless Geospatial Data Processing Workflow System Design

Pakdil

Çeli̇k

2021

IJGI

View full text Add to dashboard Cite

Geospatial data and related technologies have become an increasingly important aspect of data analysis processes, with their prominent role in most of them. Serverless paradigm have become the most popular and frequently used technology within cloud computing. This paper reviews the serverless paradigm and examines how it could be leveraged for geospatial data processes by using open standards in the geospatial community. We propose a system design and architecture to handle complex geospatial data processing jobs with minimum human intervention and resource consumption using serverless technologies. In order to define and execute workflows in the system, we also propose new models for both workflow and task definitions models. Moreover, the proposed system has new Open Geospatial Consortium (OGC) Application Programming Interface (API) Processes specification-based web services to provide interoperability with other geospatial applications with the anticipation that it will be more commonly used in the future. We implemented the proposed system on one of the public cloud providers as a proof of concept and evaluated it with sample geospatial workflows and cloud architecture best practices.

show abstract

Serverless Architecture for Big Data Analytics

Cited by 11 publications

References 7 publications

Evaluating Serverless Architecture for Big Data Enterprise Applications

Evaluating Serverless Architecture for Big Data Enterprise Applications

MapReduce based on serverless platforms

Serverless Geospatial Data Processing Workflow System Design

Contact Info

Product

Resources

About