Proactive Container Auto-scaling for Cloud Native Machine Learning Services

Buchaca, David; Berral, Josep Lluís; Wang, Chen; Youssef, Alaa

doi:10.1109/cloud49709.2020.00070

Cited by 16 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, the proposed model's ability to predict a burst increase in the needed resources remains unknown. Buchaca et al used the AI4DL framework [19], [20] to characterize workload and discover resource consumption phases. First, the existing technology was advanced to an incremental phase discovery method that applies to more general ML workload types for training and inference.…”

Section: B Proactive Scaling Trendsmentioning

confidence: 99%

Proactive Auto-Scaling Approach of Production Applications Using an Ensemble Model

2023

View full text Add to dashboard Cite

The resource usage behaviors of application workloads are currently the primary concern of cloud providers offering hosting services. These services should be able to adapt to workload changes by automatically provisioning and de-provisioning resources so that, at all times, the existing resources in a system match the current service demand. Such behavior can be achieved manually by hiring a DevOps team to manage the application's resources. Another option would be automating the resource provisioning processing using automated rules. Once such rules are met, the hosting environment will scale the resources accordingly. However, managing a DevOps team or creating flaky rules can lead to overscaling application resources. This work proposes a new approach: a proactive auto-scaling framework built on an ensemble model. Such a model utilizes several machine learning techniques to scale application resources to match resource demand before the need arises. We evaluated our solution against three real production applications hosted on Cegedim Cloud Hosting Environment, an industrial environment serving several cloud applications from various domains, and against other machine learning models used in similar proactive auto-scaling experiments mentioned in past work. The experimentation results show that predicting application resources like CPU or RAM is feasible. Moreover, even in production environments, our ensemble model performs optimally in the CPU case and is near the optimal model when predicting RAM resources.

show abstract

Section: B Proactive Scaling Trendsmentioning

confidence: 99%

Proactive Auto-Scaling Approach of Production Applications Using an Ensemble Model

2023

View full text Add to dashboard Cite

show abstract

“…Although there are approaches which take stream arrival rates into account [15], [16], they often apply model predictive control principles to predict upcoming load intensities and scale the parallelism of applied streaming operators. The prediction models are either trained on historical data from completed containers [17] or by feedback loop based system which rely on re-calibration processes in case of performance degradation [18]. In contrast, we focus on a single and efficient profiling process which enables the adaptive adjustment of resources based on the initially created model.…”

Section: Related Workmentioning

confidence: 99%

Efficient Runtime Profiling for Black-box Machine Learning Services on Sensor Streams

Becker¹,

Scheinert²,

Schmidt³

et al. 2022

Preprint

View full text Add to dashboard Cite

In highly distributed environments such as cloud, edge and fog computing, the application of machine learning for automating and optimizing processes is on the rise. Machine learning jobs are frequently applied in streaming conditions, where models are used to analyze data streams originating from e.g. video streams or sensory data. Often the results for particular data samples need to be provided in time before the arrival of next data. Thus, enough resources must be provided to ensure the just-in-time processing for the specific data stream.This paper focuses on proposing a runtime modeling strategy for containerized machine learning jobs, which enables the optimization and adaptive adjustment of resources per job and component. Our black-box approach assembles multiple techniques into an efficient runtime profiling method, while making no assumptions about underlying hardware, data streams, or applied machine learning jobs. The results show that our method is able to capture the general runtime behaviour of different machine learning jobs already after a short profiling phase.

show abstract

Reinforcement Learning Based Vertical Scaling for Hybrid Deployment in Cloud Computing

Cao

Yang

2023

Communications in Computer and Information Science

View full text Add to dashboard Cite

Proactive Container Auto-scaling for Cloud Native Machine Learning Services

Cited by 16 publications

References 15 publications

Proactive Auto-Scaling Approach of Production Applications Using an Ensemble Model

Proactive Auto-Scaling Approach of Production Applications Using an Ensemble Model

Efficient Runtime Profiling for Black-box Machine Learning Services on Sensor Streams

Reinforcement Learning Based Vertical Scaling for Hybrid Deployment in Cloud Computing

Contact Info

Product

Resources

About