Online Learning-Based Downlink Transmission Coordination in Ultra-Dense Millimeter Wave Heterogeneous Networks

Abstract

In heterogeneous ultra-dense networks with millimeter wave macro cells and small cells, base stations (BSs) and mobile user equipments (UEs) perform beamforming operations to establish highly directional links. In spite of the spatial diversity achieved through directional links, as a number of BSs are densely deployed, inter-cell interference caused by concurrent directional transmissions of adjacent BSs becomes severe, resulting in downlink performance degradation in the network. However, it is very difficult to manage inter-cell interference because of the nature of the time-varying wireless fading environment, the dynamic changes in beam propagation directivity, and unpredictable UEs' locations. In this paper, we propose an online learning-based transmission coordination algorithm based on the framework of multi-armed bandits to learn the unknown characteristics of inter-BS interference and exploit learned data to derive an optimal policy for maximizing the number of successful downlink transmissions. Through the numerical simulations, we verify the effectiveness of the proposed online learning-based inter-BS interference management scheme.

Online reinforcement learning for adaptive interference coordination

“…Regarding the dynamic nature of the network conditions (eg, traffic intensity, number of users, user positions), most previous works 7,8,14‐17,20,21,29‐34 assume static or stationary conditions. While some works consider nonstationary scenarios, 9,18,19,22,28 their approaches are based on optimization or heuristic algorithms over mathematical models of the network , like in References 6‐8,10,20,21,32,34‐37. For mathematical tractability, even the most complete mathematical network models comprise simplifications and assumptions that may limit their application to real operating networks.…”

“…RL algorithms have been previously considered for this task, mainly under stationary network conditions 16,17 but also considering nonstationary traffic conditions 23,37 as in our approach. The main issue with this approach is the well‐known curse of dimensionality , that is, the exponential growth in complexity as the state and action spaces increase.…”

Deep Reinforcement Learning for 5G Networks: Joint Beamforming, Power Control, and Interference Coordination

“…The industry standards adopted the method of almost blank subframe (ABS) to resolve the co-channel inter-cell interference problem in LTE where two base stations (BSs) interfere with one another [8]. While ABS works well in fixed beam antenna patterns, the dynamic nature of beamforming reduces the usefulness of ABS [9]. An online learning algorithm for link adaptation in multiple-input multiple-output (MIMO) bearers was studied in [2].…”

Joint Beamforming, Power Allocation, and Splitting Control for SWIPT-Enabled IoT Networks with Deep Reinforcement Learning and Game Theory

“…As a related work for LTE, the almost blank subframe (ABS) method was proposed in the standard [ 33 ] to resolve the co-channel inter-cell interference problem caused by two LTE base stations interfering with each other. Although ABS works well in fixed beam patterns, it was shown in [ 34 ] that ABS would be inefficient due to the dynamic nature of beamforming. Apart from the standard’s solution, particular attention has also been paid to the efforts on resolving different resource allocation (RA) problems.…”