Towards Demystifying Serverless Machine Learning Training

Jiang, Jiawei; Gan, Shaoduo; Liu, Yue; Wang, Fanlin; Alonso, Gustavo; Klimovic, Ana; Singla, Ankit; Wu, Wentao; Zhang, Ce

doi:10.1145/3448016.3459240

Cited by 64 publications

(31 citation statements)

References 50 publications

(54 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They found the most challenging issues of using serverless computing runtimes for training machine learning models to be their ephemerality, statelessness, and warm-up latency. In [13], Jiang et al present a comparative study of distributed ML training over FaaS and IaaS. They found that serverless training is only cost-effective with models that have a reduced communication overhead and quick convergence.…”

Section: Related Workmentioning

confidence: 99%

MLProxy: SLA-Aware Reverse Proxy for Machine Learning Inference Serving on Serverless Computing Platforms

Mahmoudi¹,

Khazaei²

2022

Preprint

View full text Add to dashboard Cite

Serving machine learning inference workloads on the cloud is still a challenging task on the production level. Optimal configuration of the inference workload to meet SLA requirements while optimizing the infrastructure costs is highly complicated due to the complex interaction between batch configuration, resource configurations, and variable arrival process. Serverless computing has emerged in recent years to automate most infrastructure management tasks.Workload batching has revealed the potential to improve the response time and cost-effectiveness of machine learning serving workloads. However, it has not yet been supported out of the box by serverless computing platforms. Our experiments have shown that for various machine learning workloads, batching can hugely improve the system's efficiency by reducing the processing overhead per request.In this work, we present MLProxy, an adaptive reverse proxy to support efficient machine learning serving workloads on serverless computing systems. MLProxy supports adaptive batching to ensure SLA compliance while optimizing serverless costs. We performed rigorous experiments on Knative to demonstrate the effectiveness of MLProxy. We showed that MLProxy could reduce the cost of serverless deployment by up to 92% while reducing SLA violations by up to 99% that can be generalized across state-of-the-art model serving frameworks.

show abstract

Section: Related Workmentioning

confidence: 99%

MLProxy: SLA-Aware Reverse Proxy for Machine Learning Inference Serving on Serverless Computing Platforms

Mahmoudi¹,

Khazaei²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…It also enables implementing truly distributed data processing operators such as joins instead of using today's contrived solutions which need to communicate through storage [24,25]. Similar ideas apply to ML over serverless, which today is expensive due to the lack of communication [19]. Boxer can also be used to implement a form of work stealing among functions of a serverless applications, since functions could communicate directly with one another to request additional work if they are idle.…”

Section: Opportunitiesmentioning

confidence: 99%

Short-lived Datacenter

Wawrzoniak¹,

Müller²,

Bruno³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Serverless platforms have attracted attention due to their promise of elasticity, low cost, and fast deployment. Instead of using a fixed virtual machine (VM) infrastructure, which can incur considerable costs to operate and run, serverless platforms support short computations, triggered on demand, with cost proportional to fine-grain function execution time. However, serverless platforms offer a restricted execution environment. For example, functions have limited execution times, limited resources, and no support for networking between functions. In this paper, we explore what it takes to treat serverless platforms as short-lived, general purpose datacenters which can execute unmodified existing applications. As a first step in this quest, we have developed Boxer, a system providing an execution environment on top of existing functions-as-a-service platforms that allows users to seamlessly migrate conventional VM-based cloud services to serverless platforms. Boxer allows generic applications to benefit from the fine-grain elasticity of serverless platforms without having to modify applications to adopt a restrictive event-triggered programming model or orchestrate auxiliary systems for data communication. We implement Boxer on top of AWS Lambda and extend it to transparently provide standard network interfaces. We describe its implementation and demonstrate how it can be used to run off-the-shelf cloud applications with a degree of fine-grained elasticity not available on traditional VM-based platforms.

show abstract

“…Serverless Machine Learning. The majority of the previous works [28], [29], [30] in this domain have focused on distributed ML training using FaaS functions. Siren [28], allows users to train ML models in the cloud using finegrained functions, thereby removing the burden of non-trivial cluster provisioning and management from the developers.…”

Section: Related Workmentioning

confidence: 99%

“…It provides a lightweight worker runtime for the cloud functions that support various ML models. Jiang et al [30] analyze the cost-performance trade-offs between Infrastructure-as-a-Service (IaaS) and FaaS for distributed ML. Towards this, they developed LambdaML, which supports different distributed ML variants such as synchronous vs. asynchronous training, purely FaaS-based, or a hybrid (FaaS/IaaS) training approach.…”

Section: Related Workmentioning

confidence: 99%

“…New FL strategies other than FedAvg can easily be implemented by modifying how the controller samples clients for each round and writing a custom aggregator function, taking the one provided as a blueprint. Having the parameter server physically close to the aggregation process is a crucial advantage of our approach due to minimized networking overhead, as is also presented in [30]. Although we deploy the aggregator function using OpenWhisk as shown in Fig 3, it can be easily deployed using another FaaS platform.…”

Section: A System Designmentioning

confidence: 99%

See 1 more Smart Citation

FedLess: Secure and Scalable Federated Learning Using Serverless Computing

Grafberger¹,

Chadha²,

Jindal³

et al. 2021

Preprint

View full text Add to dashboard Cite

The traditional cloud-centric approach for Deep Learning (DL) requires training data to be collected and processed at a central server which is often challenging in privacysensitive domains like healthcare. Towards this, a new learning paradigm called Federated Learning (FL) has been proposed that brings the potential of DL to these domains while addressing privacy and data ownership issues. FL enables remote clients to learn a shared ML model while keeping the data local. However, conventional FL systems face several challenges such as scalability, complex infrastructure management, and wasted compute and incurred costs due to idle clients. These challenges of FL systems closely align with the core problems that serverless computing and Function-as-a-Service (FaaS) platforms aim to solve. These include rapid scalability, no infrastructure management, automatic scaling to zero for idle clients, and a payper-use billing model. To this end, we present a novel system and framework for serverless FL, called FedLess. Our system supports multiple commercial and self-hosted FaaS providers and can be deployed in the cloud, on-premise in institutional data centers, and on edge devices. To the best of our knowledge, we are the first to enable FL across a large fabric of heterogeneous FaaS providers while providing important features like security and Differential Privacy. We demonstrate with comprehensive experiments that the successful training of DNNs for different tasks across up to 200 client functions and more is easily possible using our system. Furthermore, we demonstrate the practical viability of our methodology by comparing it against a traditional FL system and show that it can be cheaper and more resourceefficient.

show abstract

Towards Demystifying Serverless Machine Learning Training

Cited by 64 publications

References 50 publications

MLProxy: SLA-Aware Reverse Proxy for Machine Learning Inference Serving on Serverless Computing Platforms

MLProxy: SLA-Aware Reverse Proxy for Machine Learning Inference Serving on Serverless Computing Platforms

Short-lived Datacenter

FedLess: Secure and Scalable Federated Learning Using Serverless Computing

Contact Info

Product

Resources

About