Evaluating the Use of Policy Gradient Optimization Approach for Automatic Cloud Resource Provisioning

Funika, Włodzimierz; Koperek, Paweł

doi:10.1007/978-3-030-43229-4_40

Cited by 7 publications

(8 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such a model is used to demonstrate the performance and adaptiveness of the discussed approach to the control of dynamic traffic. In [ 10 ] we demonstrated how a similar algorithm, the Proximal Policy Optimization (PPO) [ 20 ], can be used to horizontally scale cloud resources. The implementation has been limited to control resources of a single type.…”

Section: Related Workmentioning

confidence: 99%

“…In our previous research [ 10 ] we experimented with a number of policy gradient methods (Vanilla Policy Gradient, Proximal Policy Optimization, Trust-Region Policy Optimization) out of which the PPO rendered the best empirical results in the automated resources management.…”

Section: Related Workmentioning

confidence: 99%

“…The simulator has been implemented following the results of our prior research [ 10 ]. The main process utilizes the CloudSim Plus simulation framework [ 8 ].…”

Section: Policy Trainingmentioning

confidence: 99%

“…In our previous work [ 10 ] we have demonstrated how to leverage the described ideas to create a system capable of automatic scaling of homogeneous cloud infrastructure hosting a CPU-intensive workload. In this paper we extend this approach to heterogeneous cloud resources: the system can adjust not only the amount of resources but can also decide on the features of the added resources.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Management of Heterogeneous Cloud Resources with Use of the PPO

Funika

Koperek

Kitowski

2021

Euro-Par 2020: Parallel Processing Workshops

Self Cite

View full text Add to dashboard Cite

Reinforcement learning has been recently a very active field of research. Thanks to combining it with Deep Learning, many newly designed algorithms improve the state of the art. In this paper we present the results of our attempt to use the recent advancements in Reinforcement Learning to automate the management of heterogeneous resources in an environment which hosts a compute-intensive evolutionary process. We describe the architecture of our system and present evaluation results. The experiments include autonomous management of a sample workload and a comparison of its performance to the traditional automatic management approach. We also provide the details of training of the management policy using the Proximal Policy Optimization algorithm. Finally, we discuss the feasibility to extend the presented approach to other scenarios.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

“…The simulator has been implemented following the results of our prior research [ 10 ]. The main process utilizes the CloudSim Plus simulation framework [ 8 ].…”

Section: Policy Trainingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Management of Heterogeneous Cloud Resources with Use of the PPO

Funika

Koperek

Kitowski

2021

Euro-Par 2020: Parallel Processing Workshops

Self Cite

View full text Add to dashboard Cite

show abstract

“…Reinforcement Learning [11] is a data-driven approach for adaptively applying optimized control policies based on real-time feedback, which models the stochastic process under the framework of Markov Decision Process (MDP) [21]. Policy gradient [7] is one of the most common types of reinforcement learning algorithms. In the policy gradient approach, the optimal actions with model parameters can be learned directly.…”

Section: Policy Gradient Approachmentioning

confidence: 99%

A Reinforcement Learning Based Approach to Identify Resource Bottlenecks for Multiple Services Interactions in Cloud Computing Environments

Semmes

et al. 2021

Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

View full text Add to dashboard Cite

Cloud service providers are provisioning resources including a variety of virtual machine instances to support customers that migrate their services to the cloud. From the customers' perspective, selecting the appropriate amount of resources is tightly coupled with performance and cost. By identifying the potential resource bottlenecks in the early stage of the service deployment process, resource planning can be significantly optimized. However, due to the unpredictable workloads and heterogeneous resources, it is difficult to identify resource bottlenecks that can degrade system performance. To support system non-functional requirements (NFR) in a better manner, we propose a reinforcement learning based approach to support the NFR management of system concerning the multiple services interactions scenario by identifying the potential resource bottleneck and optimizing the demanded resources. The proposed approach can predict the resource bottleneck for multiple services interactions, e.g. bottleneck in CPU or overloads in specific service, and provide guidance for resource planning. We modeled and simulated the proposed approach using an extended version of the CloudSim toolkit. Comprehensive evaluations with realistic use case from Siemens Digital Industries Software's MindSphere Solution on AliCloud show that our proposed approach can achieve high accuracy in terms of performance metrics, such as response time, queries per second (QPS), and resource usage.

show abstract

Continuous self‐adaptation of control policies in automatic cloud management

Funika

Koperek

Kitowski

2022

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

Deep reinforcement learning has been recently a very active field of research. The policies generated with the use of this class of training algorithms are flexible and thus have many practical applications. In this article we present the results of our attempt to use the recent advancements in reinforcement learning to automate the management of resources in a compute cloud environment. We describe a new approach to self-adaptation of autonomous management, which uses a digital clone of the managed infrastructure to continuously update the control policy. We present the architecture of our system and discuss the results of evaluation which includes autonomous management of a sample application deployed to Amazon Web Services cloud. We also provide the details of the training of the management policy using the Proximal Policy Optimization algorithm. Finally, we discuss the feasibility to extend the presented approach to further scenarios.

show abstract

Evaluating the Use of Policy Gradient Optimization Approach for Automatic Cloud Resource Provisioning

Cited by 7 publications

References 10 publications

Management of Heterogeneous Cloud Resources with Use of the PPO

Management of Heterogeneous Cloud Resources with Use of the PPO

A Reinforcement Learning Based Approach to Identify Resource Bottlenecks for Multiple Services Interactions in Cloud Computing Environments

Continuous self‐adaptation of control policies in automatic cloud management

Contact Info

Product

Resources

About