Reinforcement Learning vs. Computational Intelligence: Comparing Service Management Approaches for the Cloud Continuum

Abstract

Modern computing environments, thanks to the advent of enabling technologies such as Multi-access Edge Computing (MEC), effectively represent a Cloud Continuum, a capillary network of computing resources that extend from the Edge of the network to the Cloud, which enables a dynamic and adaptive service fabric. Efficiently coordinating resource allocation, exploitation, and management in the Cloud Continuum represents quite a challenge, which has stimulated researchers to investigate innovative solutions based on smart techniques such as Reinforcement Learning and Computational Intelligence. In this paper, we make a comparison of different optimization algorithms and a first investigation of how they can perform in this kind of scenario. Specifically, this comparison included the Deep Q-Network, Proximal Policy Optimization, Genetic Algorithms, Particle Swarm Optimization, Quantum-inspired Particle Swarm Optimization, Multi-Swarm Particle Optimization, and the Grey-Wolf Optimizer. We demonstrate how all approaches can solve the service management problem with similar performance—with a different sample efficiency—if a high number of samples can be evaluated for training and optimization. Finally, we show that, if the scenario conditions change, Deep-Reinforcement-Learning-based approaches can exploit the experience built during training to adapt service allocation according to the modified conditions.

Edge and Fog Computing for the Internet of Things

“…Finally, the fourth paper [4] focuses on crucial architectural aspects like resource allocation, exploitation, and management in the Cloud Continuum, as well as a way to indicate the capillary network that interconnects the edge and the cloud, thereby enabling dynamic and adaptive services. In this context, the paper analyses a number of techniques in the domain of reinforcement learning and computational intelligence, with the aim of solving a service management problem by dealing with the optimization of the services allocated in the Cloud Continuum itself.…”