Benchmarking, analysis, and optimization of serverless function snapshots

Abstract

Serverless computing has seen rapid adoption due to its high scalability and flexible, pay-as-you-go billing model. In serverless, developers structure their services as a collection of functions, sporadically invoked by various events like clicks. High inter-arrival time variability of function invocations motivates the providers to start new function instances upon each invocation, leading to significant cold-start delays that degrade user experience. To reduce cold-start latency, the industry has turned to snapshotting, whereby an image of a fully-booted function is stored on disk, enabling a faster invocation compared to booting a function from scratch.This work introduces vHive, an open-source framework for serverless experimentation with the goal of enabling researchers to study and innovate across the entire serverless stack. Using vHive, we characterize a state-of-the-art snapshot-based serverless infrastructure, based on industry-leading Containerd orchestration framework and Firecracker hypervisor technologies. We find that the execution time of a function started from a snapshot is 95% higher, on average, than when the same function is memoryresident. We show that the high latency is attributable to frequent page faults as the function's state is brought from disk into guest memory one page at a time. Our analysis further reveals that functions access the same stable working set of pages across different invocations of the same function. By leveraging this insight, we build REAP, a light-weight software mechanism for serverless hosts that records functions' stable working set of guest memory pages and proactively prefetches it from disk into memory. Compared to baseline snapshotting, REAP slashes the cold-start delays by 3.7×, on average. CCS CONCEPTS• Computer systems organization → Cloud computing; • Information systems → Computing platforms; Data centers; • Software and its engineering → n-tier architectures.

Serverless computing on heterogeneous computers

“…Fork-based startup [51,54,78] and snapshot-based startup [11, 16,35,54,91,92] are the two most widely adopted optimizations for reducing startup latency. Molecule follows the line of research, with two new contributions.…”

“…As shown in Figure 15 (a), snapshot (or checkpoint and restore) and fork are the two most widely adopted optimizations for reducing startup latency. For example, Replayable Execution [92] and FireCracker [91] leverage prepared snapshots to mitigate the application initialization cost. Catalyzer leverages With Molecule, these steps are sufficient to enable GPU for serverless computing, and GPU functions can seamlessly cooperate with CPU, DPU and FPGA functions.…”

“…System optimizations for serverless computing. Prior works propose many optimizations for serverless computing, including snapshot and fork-based startup [51,54,54,78,91,92], IPC-based communication [43,61], and others [57,63,70,74,88,94,95]. Molecule follows the line of research and proposes cfork and nIPC-based communication to achieve low startup and communication latency on heterogeneous computers.…”

“…There are already many implementations [4,9,18,42] and optimizations [11,16,35,43,54,61,78,84,91,92] of serverless systems. However, (almost) all of these works focus on serverless computing with a homogeneous computer, one that contains processing units (to run functions) with the same computation ability (typically same ISA and frequency).…”

MAGE: Nearly Zero-Cost Virtual Memory for Secure Computation

“…Thus, we use our staged approach ( §6) to find a good approximation. Some systems use observations of past memory accesses or past working sets (e.g., from prior invocations of a program) to perform targeted prefetching [33,35,56,77,92] and approx-imate Belady's algorithm (MIN) [72]. SC's obliviousness and our memory programming approach allow MAGE to compute the memory access pattern without first running the program, and then apply these techniques using the access pattern itself.…”