Mitigating Cold Start Problem in Serverless Computing with Function Fusion

Abstract

[sangyoon]As Artificial Intelligence (AI) is becoming ubiquitous in many applications, serverless computing is also emerging as a building block for developing cloud-based AI services. Serverless computing has received much interest because of its simplicity, scalability, and resource efficiency. However, due to the trade-off with resource efficiency, serverless computing suffers from the cold start problem, that is, a latency between a request arrival and function execution[sangyoon] that is encountered due to resource provisioning. [sangyoon]In serverless computing, functions can be composed as workflows to process a complex task, and the cold start problem has a significant influence on workflow response time because the cold start can occur in each function.The cold start problem significantly influences the overall response time of workflow that consists of functions because the cold start may occur in every function within the workflow. Function fusion can be one of the solutions to mitigate the cold start latency of a workflow. If two functions are fused into a single function, the cold start of the second function is removed; however, if parallel functions are fused, the workflow response time can be increased because the parallel functions run sequentially even if the cold start latency is reduced. This study presents an approach to mitigate the cold start latency of a workflow using function fusion while considering a parallel run. First, we identify three latencies that affect response time, present a workflow response time model considering the latency, and efficiently find a fusion solution that can optimize the response time on the cold start. Our method shows a response time of 28–86% of the response time of the original workflow in five workflows.

Managing Cold-start in The Serverless Cloud with Temporal Convolutional Networks

“…Specifically, a model can set functions' pre-warm time and keep-alive time based on dependency-predictable functions' invocation histograms, while using fixed-timeout policy for dependency-unpredictable functions [19]. Function-level cold-start problems can also be manged by reducing application code loading latency by loading critical functions first and optional functions later [20], or by fusing and optimizing function codes for more efficient run-time execution [21], [22].…”

“…Finergrain code dependencies can be inferred and pass to path A to assist with future demand inferences. Function codes can then be optimized per [21], [22]. Based on function instances' arrival time, fine-grain requirements, and optimized function code graphs, instances can be prioritized.…”

µFuncCache: A User-Side Lightweight Cache System for Public FaaS Platforms

“…Before each test, we called the function chain several times to ensure that all cloud functions have live instances, in order to avoid the impact of a cold start (i.e., set-up time required that is invoked when downloading the code, starting the instance, and initializing the runtime, and so on, for the first time) [3,6], which has been studied extensively with some work also focusing on the optimization for the chain structure [24,25]. The call chain length was set from 1 to 4 and, for each run, we conducted 20 tests and calculated the average time.…”