Deep Reinforcement Learning for Dynamic Network Slicing in IEEE 802.11 Networks

Abstract

Network slicing, a key enabler for future wireless networks, divides a physical network into multiple logical networks that can be dynamically created and configured. In current IEEE 802.11 (Wi-Fi) networks, the only form of network configuration is a rule-based optimization of few parameters. Future access points (APs) are expected to have self-organizational capabilities, able to deal with large configuration spaces in order to dynamically configure each slice. Deep Reinforcement Learning (DRL) can achieve promising results in highly dynamic and complex environments without the need for an operating model, by learning the optimal strategy after interacting with the environment. However, since the number of possible slice configurations is huge, achieving the optimal strategy requires an exhaustive learning period that might yield an outdated slice configuration. In this paper, we propose a fast-learning DRL model that can dynamically optimize the slice configuration of unplanned Wi-Fi networks without expert knowledge. Enhanced with an off-line learning step, the proposed approach is able to achieve the optimal slice configuration with a fast convergence, which is attractive for dynamic scenarios.

Survey on Network Slicing for Internet of Things Realization in 5G Networks

“…The proposed RLCO algorithm was able to solve the problems of poor efficiency of existing algorithms in virtual network mapping, low resource utilisation and poor coordination between node mapping and link mapping. In [234], Bast et al proposed a fast-learning Deep Reinforcement Learning (DRL) model that has the ability to optimise the slice configuration of unplanned Wi-Fi networks dynamically without expert knowledge. The proposed approach was able to optimise various Wi-Fi parameters per slice dynamically.…”

Applying Deep Reinforcement Learning to Improve Throughput and Reduce Collision Rate in IEEE 802.11 Networks

“…Sun et al [17] proposed an autonomous energy management framework using cell activation techniques and designed a Q-learning model with reduced state space size to consider varying resource demand and user population. Further, some studies [19], [20] adopted DRL to optimize various Wi-Fi parameters in highly dynamic and complex environments. Balakrishnan et al [19] adopted DRL to the problem of allocating time and frequency resources in OFDMA wireless systems.…”

“…Balakrishnan et al [19] adopted DRL to the problem of allocating time and frequency resources in OFDMA wireless systems. Bast et al [20] adopted DRL model that can dynamically optimize the slice configuration of unplanned Wi-Fi networks without expert knowledge.…”

“…Zhang et al [18] explained that deep RL (DRL) uses deep neural networks (DNN) to deal with more complex environments. Furthermore, some studies [19], [20] adopt DRL to optimize various Wi-Fi parameters in highly dynamic and complex environments. In addition, the RL and DRL algorithms can now be used to study CW optimization because of the high computing capabilities in modern network devices.…”

Stateless Reinforcement Learning for Multi-Agent Systems: the Case of Spectrum Allocation in Dynamic Channel Bonding WLANs

“…We then justify why stateless RL formulations and, in particular, multi-armed bandits (MABs) are better suited to the problem over temporal difference variations like Q-learning. As well, we call into question the usefulness of deep reinforcement learning (DRL) due to the unfeasible amount of data needed to learn [6].…”