Deep Reinforcement Learning for QoS-Constrained Resource Allocation in Multiservice Networks

Abstract

In this article, we study a Radio Resource Allocation (RRA) that was formulated as a non-convex optimization problem whose main aim is to maximize the spectral efficiency subject to satisfaction guarantees in multiservice wireless systems. This problem has already been previously investigated in the literature and efficient heuristics have been proposed. However, in order to assess the performance of Machine Learning (ML) algorithms when solving optimization problems in the context of RRA, we revisit that problem and propose a solution based on a Reinforcement Learning (RL) framework. Specifically, a distributed optimization method based on multi-agent deep RL is developed, where each agent makes its decisions to find a policy by interacting with the local environment, until reaching convergence. Thus, this article focuses on an application of RL and our main proposal consists in a new deep RL based approach to jointly deal with RRA, satisfaction guarantees and Quality of Service (QoS) constraints in multiservice celular networks. Lastly, through computational simulations we compare the state-of-art solutions of the literature with our proposal and we show a near optimal performance of the latter in terms of throughput and outage rate.

Optimized intellectual resource scheduling using deep reinforcement Q‐learning in cloud computing

“…The efficiency of this technique is mainly evaluated by its ability to minimize delay and energy consumption. Saraiva et al 40 developed a model using DRL for QoS constrained resource allocation in multiservice networks. This model offers improved performance in terms of throughput and outage rate.…”

5GhNet: an intelligent QoE aware RAT selection framework for 5G-enabled healthcare network

“…Then, Adam optimisation is employed to update online network weights on the basis of F( ) i.e. loss function and defined as Adam optimisation is preferred over stochastic gradient descent algorithm in case of sparse gradient problems as it calculates independent adaptive learning rate (Saraiva et al 2020;Bhattacharya et al 2019) and efficiently reduces the loss function described in Eq. ( 26).…”

QAAs: QoS provisioned artificial intelligence framework for AP selection in next-generation wireless networks

“…Therefore, the network state S t perceived through southbound interface is preprocessed to φ(S t ) and while every mapping, Q network generates a segment of memory comprising current state φ(S t ), current action a t , instant reward r t , and next state φ(S t+1 ) that are stacked in experience buffer D during the training process to improve the learning efficiency. In this way, the experience buffer strategy reduces the correlation among the training examples in order to avoid the divergence of optimal policy to the local minimum [27]. The choice of mini-batch size plays an important role in tuning deep learning systems.…”

“…Adam optimisation algorithm has been adopted to update network weights iterative based on training data. In comparison to the stochastic gradient descent algorithm, Adam optimisation computes individual adaptive learning rates rather maintaining a single learning rate which improves the performance on sparse gradient problems and minimises mean square error E(θ ) [27,45]. The overall algorithm of agent module is divided into two parts, i.e.…”

“…Unfortunately, the above discussed works fail to guarantee the user QoS requirements in a dense WLAN environment. Saraiva et al [27] proposed a reinforcement learning solution for the radio resource allocation, formulated as a non-convex optimisation problem to guarantee user satisfaction in QoS constrained cellular environment. Ding et al [28] leveraged multi-agent deep reinforcement learning for joint user association and power allocation as well as ensuring user QoS requirements in OFDMA based hetnet.…”