Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Abstract

This paper presents a comprehensive literature review on applications of deep reinforcement learning in communications and networking. Modern networks, e.g., Internet of Things (IoT) and Unmanned Aerial Vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, deep reinforcement learning, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of deep reinforcement learning from fundamental concepts to advanced models. Then, we review deep reinforcement learning approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks such as 5G and beyond. Furthermore, we present applications of deep reinforcement learning for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying deep reinforcement learning.

Machine Learning for Resource Management in Cellular and IoT Networks: Potentials, Current Solutions, and Open Challenges

“…Authors provided comprehensive literature survey on deep learning in wireless communication networks followed by various case-studies, in which DL has been useful. In the same spirit, authors presented and discussed various advance level deep Q learning (DQL) models in [111]. DQL inherits the advantages of DL and Q-learning and find its use fullness in applications having large state and action spaces.…”

A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access

“…Inspired by the achievements of reinforcement learning in dynamic control problems, such as the game of Atari [16], and AlphaGo [17], there has been increased interest in seeking reinforcement learning based solutions for problems in wireless communications. As summarized in [18] and [19], deep reinforcement learning algorithms have been applied in various wireless settings. For example, the authors in [20] and [21] investigate the use of Q-learning and SARSA (state-action-reward-stateaction) reinforcement learning, respectively, in power control.…”

A Reinforcement Learning Approach for the Multichannel Rendezvous Problem

“…To address such a question, we adopt a reinforcement learning approach to learn the channel selection probabilities of a SU. Reinforcement learning (see, e.g., the book [13] and the recent survey [14]) is a field of machine learning that addresses the problems of how to behave in an environment by performing certain actions and observing the reward from those actions. In these problems, the fixed limited resources must be allocated to maximize their expected gain.…”