FaaSRank: Learning to Schedule Functions in Serverless Platforms

Yu, Hanfei; Irissappane, Athirai A.; Wang, Hao; Lloyd, Wes

doi:10.1109/acsos52086.2021.00023

Cited by 12 publications

(4 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous works showed that the setup time of worker nodes and related supporting infrastructure is expensive [8], [15]. The solutions include preemptively provision network cache and shell-nodes [15], preemptively provision new shell nodes and load-balance existing worker nodes across physical clusters [8], rank and select servers for function scheduling [16], and preemptively provision the worker nodes based on node cache miss rate [17]. Keep-alive policies can be based on operational priorities such as cost, frequencies, time, size [17].…”

Section: A Previous Workmentioning

confidence: 99%

“…If future demands will not be met, more infrastructure supports and/or shell nodes will be preemptively provisioned per [6], [8], [12], [15], [18]. If future demands can be met, the system will rank existing available resources and orchestrate them per [16]. Resource ranking and infra-structure orchestration schedules will be shared with path B.…”

Section: B the Ensemble Policymentioning

confidence: 99%

See 1 more Smart Citation

Managing Cold-start in The Serverless Cloud with Temporal Convolutional Networks

Tn¹

2023

Preprint

View full text Add to dashboard Cite

Serverless cloud is an innovative cloud service model that frees customers from most cloud management duties. It also offers the same advantages as other cloud models but at much lower costs. As a result, the serverless cloud has been increasingly employed in high-impact areas such as system security, banking, and health care. A big threat to the serverless cloud's performance is cold-start, which is when the time of provisioning the needed cloud resource to serve customers' requests incurs unacceptable costs to the service providers and/or the customers.This paper proposes a novel low-coupling, high-cohesion ensemble policy that addresses the cold-start problem at infrastructure- and function-levels of the serverless cloud stack, while the state of the art policies have a more narrowed focus. This ensemble policy anchors on the prediction of function instance arrivals, 10 to 15 minutes into the future. It is achievable by using the temporal convolutional network (TCN) deep-learning method. Bench-marking results on a real-world dataset from a large-scale serverless cloud provider show that TCN out-performs other popular machine learning algorithms for time series. Going beyond cold-start management, the proposed policy and publicly available codes can be adopted in solving other cloud problems such as optimizing the provisioning of virtual software-defined network assets.

show abstract

Section: A Previous Workmentioning

confidence: 99%

Section: B the Ensemble Policymentioning

confidence: 99%

Managing Cold-start in The Serverless Cloud with Temporal Convolutional Networks

Tn¹

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…A policy gradient algorithm is proposed in [34], to identify the best node for scheduling a function request. [35] uses a DRL approach to determine the percentage of user requests to be processed by the cloud and offloaded to the fog layer.…”

Section: B Rl Solutions For Serverless Resource Managementmentioning

confidence: 99%

Deep reinforcement learning for application scheduling in resource-constrained, multi-tenant serverless computing environments

Mampage

Karunasekera

Buyya

2023

Future Generation Computer Systems

View full text Add to dashboard Cite

“…The loadlocality tradeoff we explore is complementary to these CPU scheduling optimizations. The repetitive nature of functions and their workflows can also be used to improve resource utilization and latency [9,19,28,37]: our load-balancer is stateless for the sake of simplicity and can be enhanced with these techniques if necessary.…”

Section: Related Workmentioning

confidence: 99%

Locality-aware Load-Balancing For Serverless Clusters

Fuerst

Sharma

2022

Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing

View full text Add to dashboard Cite

While serverless computing provides more convenient abstractions for developing and deploying applications, the Function-as-a-Service (FaaS) programming model presents new resource management challenges for the FaaS provider. In this paper, we investigate load-balancing policies for serverless clusters. Locality, i.e., running repeated invocations of a function on the same server, is a key determinant of performance because it increases warm-starts and reduces cold-start overheads. We find that the locality vs. load tradeoff is crucial and presents a large design space.We enhance consistent hashing for FaaS, and develop CH-RLU: Consistent Hashing with Random Load Updates, a simple practical load-balancing policy which provides more than 2× reduction in function latency. Our policy deals with highly heterogeneous, skewed, and bursty function workloads, and is a drop-in replacement for OpenWhisk's existing load-balancer. We leverage techniques from caching such as SHARDS for popularity detection, and develop a new approach that places functions based on a tradeoff between locality, load, and randomness. CCS CONCEPTS• Computer systems organization → Cloud computing.

show abstract

FaaSRank: Learning to Schedule Functions in Serverless Platforms

Cited by 12 publications

References 13 publications

Managing Cold-start in The Serverless Cloud with Temporal Convolutional Networks

Managing Cold-start in The Serverless Cloud with Temporal Convolutional Networks

Deep reinforcement learning for application scheduling in resource-constrained, multi-tenant serverless computing environments

Locality-aware Load-Balancing For Serverless Clusters

Contact Info

Product

Resources

About