Multi-Agent Reinforcement Learning-Based Resource Management for End-to-End Network Slicing

Abstract

To meet the explosive growth of mobile traffic, the 5G network is designed to be flexible and support multi-access edge computing (MEC), thereby improving the end-to-end quality of service (QoS). In particular, 5G network slicing, which allows a physical infrastructure to split into multiple logical networks, keeps the balance of network resource allocation among different service types with on-demand resource requests. However, achieving effective resource allocation across the end-to-end network is difficult due to the dynamic characteristics of slicing requests such as uncertain real-time resource demand and heterogeneous requirements. In this paper, we develop a reinforcement learning (RL)-based dynamic resource allocation framework for end-to-end network slicing with heterogeneous requirements in multi-layer MEC environments. We first design a hierarchical MEC architecture and formulate a resource allocation problem for the end-to-end network slicing as an optimization problem using the Markov decision process (MDP). Using proximal policy optimization (PPO), we develop independently-collaborative and jointlycollaborative dynamic resource allocation algorithms to maximize resource efficiency while satisfying the QoS of slices. Experimental results show that the proposed algorithms can recognize the characteristics of slice requests and coming resource demands and efficiently allocate resources with a high QoS satisfaction rate.INDEX TERMS 5G, network slicing, multi-access edge computing, network resource management, multiagent reinforcement learning.

Model-Based Reinforcement Learning With Kernels for Resource Allocation in RAN Slices

“…Consequently, RL is becoming a standard approach for resource management in network slicing. The main issues addressed by RL in this domain have been: resource management for virtualized functions [9], [11], [23]- [26], admission control of new slices [9], [15], [16], handling of computational resources in mobile edge computing (MEC) [11], [27], [28], UE scheduling [29], [30], and radio resource allocation among network slices [13], [18], [19], [31], [32], which is the problem addressed in this work.…”

Distributed reinforcement learning for dynamic spectrum allocation in cognitive radio‐based internet of things

“…In the last few years, RL and DRL techniques have increasingly attracted research community interest, due to their robustness and dynamicity. They have demonstrated them superlatively in the context of dynamic channel allocation, and many papers [47][48][49][50][51][52][53][54][55] have mainly based on the Q-learning, SARSA, expected SARSA, Monte Carlo, and Actor-Critic (A2C), to allocate radio or edge/fog resources to network slices, in order to maximise the operator revenue, QoS satisfaction, and resource utilisation. To deal with scalability issues faced by RL-based approaches, the approaches in Ref.…”

“…Related studies can be found in the following sources but not restricted to Ref. [7–87]. Each of the proposed techniques has its own advantages and disadvantages, and none of them can be considered on its own to fully accomplish the realisation of CR systems.…”

ICRAN: Intelligent Control for Self-Driving RAN Based on Deep Reinforcement Learning

“…maximize resource utilization while ensuring an acceptable level of performance). More specifically, the existing proposals focused on tracking and optimizing a single metric like coverage [6], power management [7], throughput [8] and resource sharing [9] at a time. It flows directly from this limited focus that current approaches resort to often applying a single control action, e.g.…”