Benchmarking parallelism in FaaS platforms

Barcelona-Pons, Daniel; Garcı́a-López, Pedro

doi:10.1016/j.future.2021.06.005

Cited by 28 publications

(21 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, serverless fulfills its promise of built-in scalability under the given load levels for our 10 applications. This result was somewhat unexpected, especially contrasting with prior research [10,38,72]. However, of course, bursty workloads may still negatively impact performance on different platforms or with even more rapid bursts than what we evaluated (e.g., per-microsecond bursts rather than per-second bursts).…”

Section: Discussioncontrasting

confidence: 99%

“…We conduct a performance benchmarking experiment [29] with an open-loop load generator in the data center region Northern Virginia (us-east-1) as commonly used by other serverless studies [10,14,16,81,82]. We collected over 7.5 million traces, through over 12 months of experimentation in 2021 and 2022.…”

Section: Experiments Designmentioning

confidence: 99%

“…Real-world applications exhibit diverse invocation patterns [66] but prior work rarely investigated dynamic workloads over time [38] or different invocation patterns [37] and if so, using artificial applications, patterns, and one-time bursts [10,72,81]. It remains unclear how different invocation patterns derived from the Azure Function Traces [66] (Section 3.4) affect the end-to-end latency of serverless applications.…”

Section: Invocation Patternsmentioning

confidence: 99%

See 2 more Smart Citations

Let's Trace It: Fine-Grained Serverless Benchmarking using Synchronous and Asynchronous Orchestrated Applications

Scheuner¹,

Eismann²,

Talluri³

et al. 2022

Preprint

View full text Add to dashboard Cite

Making serverless computing widely applicable requires detailed performance understanding. Although contemporary benchmarking approaches exist, they report only coarse results, do not apply distributed tracing, do not consider asynchronous applications, and provide limited capabilities for (root cause) analysis. Addressing this gap, we design and implement ServiBench, a serverless benchmarking suite. ServiBench (i) leverages synchronous and asynchronous serverless applications representative of production usage, (ii) extrapolates cloud-provider data to generate realistic workloads, (iii) conducts comprehensive, end-to-end experiments to capture application-level performance, (iv) analyzes results using a novel approach based on (distributed) serverless tracing, and (v) supports comprehensively serverless performance analysis. With ServiBench, we conduct comprehensive experiments on AWS, covering five common performance factors: median latency, cold starts, tail latency, scalability, and dynamic workloads. We find that the median end-to-end latency of serverless applications is often dominated not by function computation but by external service calls, orchestration, or trigger-based coordination. We release collected experimental data under FAIR principles and ServiBench as a tested, extensible open-source tool.

show abstract

Section: Discussioncontrasting

confidence: 99%

Section: Experiments Designmentioning

confidence: 99%

Section: Invocation Patternsmentioning

confidence: 99%

See 1 more Smart Citation

Let's Trace It: Fine-Grained Serverless Benchmarking using Synchronous and Asynchronous Orchestrated Applications

Scheuner¹,

Eismann²,

Talluri³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For a technology review of these and other FaaS platforms and a classification framework for them, see [56]. For a parallelism benchmark for the biggest pay-as-you-go FaaS providers, see [57].…”

Section: Architecture and Methodology A Function-as-a-servicementioning

confidence: 99%

Using a Digital Twin as the Objective Function for Evolutionary Algorithm Applications in Large Scale Industrial Processes

et al. 2023

View full text Add to dashboard Cite

In this paper, we describe how the up-to-date state of a digital twin, and its corresponding simulation model, can be used as a fitness function of an evolutionary algorithm for optimizing a large-scale industrial process. An ICT architecture is presented for solving the computational challenges that arise when the fitness function evaluation takes considerable amount of time. Parallel computation of the fitness function in a cloud computing environment is proposed and the evolutionary algorithm is connected to the computational environment using the Function-as-a-Service approach. A case-study was conducted on the district heating network of Espoo, the second largest city in Finland. The study shows that the architecture is suited for optimizing the operating costs of the large district heating network, with over 800 km of water pipes and over 14 heat producers, reaching a cost-saving of an average of 2%, and up-to 4%, over the current industrial state-of-the-art method in use at the city of Espoo.

show abstract

“…For this purpose, we used the embarrassingly parallel Monte Carlo algorithm, which is intensively used in HPC, cloud, and serverless computing [39], [40], [41], [42]. We implemented it as an FC, as presented in Fig.…”

Section: How Do Massive Functions Failures Affect Fcs?mentioning

confidence: 99%

FaaScinating Resilience for Serverless Function Choreographies in Federated Clouds

Ristov

Kimovski

Fahringer

2022

IEEE Trans. Netw. Serv. Manage.

View full text Add to dashboard Cite

Cloud applications often benefit from deployment on serverless technology Function-as-a-Service (FaaS), which may instantly spawn numerous functions and charges users for the period when serverless functions are running. Maximum benefit is achieved when functions are orchestrated in a workflow or function choreographies (FCs). However, many provider limitations specific for FaaS, such as maximum concurrency or duration often increase the failure rate, which can severely hamper the execution of entire FCs. Current support for resilience is often limited to function retries or try-catch, which are applicable within the same cloud region only. To overcome these limitations, we introduce rAF CL, a middleware platform that maintains reliability of complex FCs in federated clouds. In order to support resilient FC execution under rAF CL, our model creates an alternative strategy for each function based on the required availability specified by the user. Alternative strategies are not restricted to the same cloud region, but may contain alternative functions across five providers, invoked concurrently in a single alternative plan or executed subsequently in multiple alternative plans. With this approach, rAF CL offers flexibility in terms of cost-performance trade-off. We evaluated rAF CL by running three real-life applications across three cloud providers. Experimental results demonstrated that rAF CL outperforms the resilience of AWS Step Functions, increasing the success rate of entire FC by 53.45%, while invoking only 3.94% more functions with zero wasted function invocations. rAF CL significantly improves availability of entire FCs to almost 1 and survives even after massive failures of alternative functions.

show abstract

Benchmarking parallelism in FaaS platforms

Cited by 28 publications

References 4 publications

Let's Trace It: Fine-Grained Serverless Benchmarking using Synchronous and Asynchronous Orchestrated Applications

Let's Trace It: Fine-Grained Serverless Benchmarking using Synchronous and Asynchronous Orchestrated Applications

Using a Digital Twin as the Objective Function for Evolutionary Algorithm Applications in Large Scale Industrial Processes

FaaScinating Resilience for Serverless Function Choreographies in Federated Clouds

Contact Info

Product

Resources

About