Dealing with Limited Backhaul Capacity in Millimeter-Wave Systems: A Deep Reinforcement Learning Approach

Abstract

Millimeter Wave (MmWave) communication is one of the key technology of fifth generation (5G) wireless systems to achieve the expected 1000x data rate. With large bandwidth at mmWave band, the link capacity between users and base stations (BS) can be much higher compared to sub-6GHz wireless systems. Meanwhile, due to the high cost of infrastructure upgrade, it would be difficult for operators to drastically enhance the capacity of backhaul links between mmWave BSs and the core network. As a result, the data rate provided by backhaul may not be sufficient to support all mmWave links, the backhaul connection becomes the new bottleneck that limits the system performance. On the other hand, as mmWave channels are subject to random blockage, the data rates of mmWave users significantly vary over time. With limited backhaul capacity and highly dynamic data rates of users, how to allocate backhaul resource to each user remains a challenge for mmWave systems. In this article, we present a deep reinforcement learning (DRL) approach to address this challenge. By learning the blockage pattern, the system dynamics can be captured and predicted, resulting in efficient utilization of backhaul resource. We begin with a discussion on DRL and its application in wireless systems. We then investigate the problem backhaul resource allocation and present the DRL based solution. Finally, we discuss open problems for future research and conclude this article.

TCP-Drinc: Smart Congestion Control Based on Deep Reinforcement Learning

“…The benefits are two-fold: accelerating the training process and better explanation of the model. According to (6) and 7, the current sending rate is determined by the cWnd difference and the sending rate one RTT ago. Therefore, cWnd difference, w, as measured by the difference in two consecutive time slots, and RTT τ RTT should be collected as features.…”

“…In this paper, we aim to develop a smart congestion control algorithm that does not rely on the above assumptions. Motivated by the recent success of applying machine learning to wireless networking problems [5], and based on our experience of applying deep learning (DR) and deep reinforcement learning (DRL) to 5G mmWave networks [6], edge computing and caching [7]- [9], and RF sensing and indoor localization [10]- [12], we propose to develop a modelfree, smart congestion control algorithm based on DRL. The original methods that treat the network as a white box have been shown to have many limitations.…”

Hybrid Precoding for Multiuser Millimeter Wave Massive MIMO Systems: A Deep Learning Approach

“…We summarize the algorithmic steps of the proposed approach in Algorithm 1. Note that the proposed hybrid precoding optimization in (15) is different than the one in [7], in which, not all possible combinations of the analog precoders are considered as it is done in this work. In Section VI, we show that (15) yields better results as compared to [7].…”

Smart Power Control for Quality-Driven Multi-User Video Transmissions: A Deep Reinforcement Learning Approach

“…This is quite important for resource optimization problems in wireless networks, where the channel state changes rapidly. There is now an increasing interest on incorporating DRL into the design of wireless networking algorithms [20], such as mobile off-loading [21], dynamic channel access [22], [23], mobile edge computing and caching [24], [25], dynamic base station on and off [26], TCP congestion control [27], and resource allocation [28]- [31].…”