A Deep Reinforcement Learning-Based Resource Scheduler for Massive MIMO Networks

Abstract

The large number of antennas in massive MIMO systems allows the base station to communicate with multiple users at the same time and frequency resource with multiuser beamforming. However, highly correlated user channels could drastically impede the spectral efficiency that multi-user beamforming can achieve. As such, it is critical for the base station to schedule a suitable group of users in each time and frequency resource block to achieve maximum spectral efficiency while adhering to fairness constraints among the users. In this paper, we consider the resource scheduling problem for massive MIMO systems with its optimal solution known to be NP-hard. Inspired by recent achievements in deep reinforcement learning (DRL) to solve problems with large action sets, we propose SMART, a dynamic scheduler for massive MIMO based on the state-of-the-art Soft Actor-Critic (SAC) DRL model and the K-Nearest Neighbors (KNN) algorithm. Through comprehensive simulations using realistic massive MIMO channel models as well as real-world datasets from channel measurement experiments, we demonstrate the effectiveness of our proposed model in various channel conditions. Our results show that our proposed model performs very close to the optimal proportionally fair (Opt-PF) scheduler in terms of spectral efficiency and fairness with more than one order of magnitude lower computational complexity in medium network sizes where Opt-PF is computationally feasible. Our results also show the feasibility and high performance of our proposed scheduler in networks with a large number of users and resource blocks.

Deep Discretized Deterministic Policy Gradient Based Resource AllocationConsidering Network Slicing and Device-to-Device Paradigms

“…Considering the complexity of the resource management problem. High-dimensional tasks are generally challenging to deal with for the DRL model due to a phenomenon known as the curse of dimensionality [13]. We propose the DDPG-k algorithm to perform resource allocation because it has better learning in high-dimensional action spaces.…”

“…However, resource allocation is a discrete decision problem where the combination of the resource allocation cannot exceed the available total value. Inspired by the approach in [13], we use the KNN to discretize DDPG to adapt it to discrete action spaces. The basic idea is to use a continuous-based algorithm to generate an initial continuous action first.…”

“…The DDPG agent is based on the actor-critic method and the deterministic gradient policy where the action is a single value given by the output of the actor neural network. Several recent works have attempted to solve the problem of the some RL approaches presenting a large discrete action space by discretizing the continuous action space [13,21]. In this sense, as shown by the proposed Algorithm 1, a KNN approximation is used because of its agile search in logarithmic time and maps a continuous action space to a discrete action space [17].…”