A Comparison of Reinforcement Learning Algorithms in Fairness-Oriented OFDMA Schedulers

Abstract

Due to large-scale control problems in 5G access networks, the complexity of radioresource management is expected to increase significantly. Reinforcement learning is seen as apromising solution that can enable intelligent decision-making and reduce the complexity of differentoptimization problems for radio resource management. The packet scheduler is an importantentity of radio resource management that allocates users’ data packets in the frequency domainaccording to the implemented scheduling rule. In this context, by making use of reinforcementlearning, we could actually determine, in each state, the most suitable scheduling rule to be employedthat could improve the quality of service provisioning. In this paper, we propose a reinforcementlearning-based framework to solve scheduling problems with the main focus on meeting the userfairness requirements. This framework makes use of feed forward neural networks to map momentarystates to proper parameterization decisions for the proportional fair scheduler. The simulation resultsshow that our reinforcement learning framework outperforms the conventional adaptive schedulersoriented on fairness objective. Discussions are also raised to determine the best reinforcement learningalgorithm to be implemented in the proposed framework based on various scheduler settings.

Actor-Critic Learning Based QoS-Aware Scheduler for Reconfigurable Wireless Networks

“…An ML-based RRM algorithm is proposed in [27] to estimate the required resources in the network for tackling traffic on-demand traffics over HTTP. ML has been used in resource allocation by considering various QoS combinations objectives such as packet loss [23], delay [24], and user fairness [25]. However, these models are defined for improving delay or enhancing throughput for Ultra-Reliable and Low-Latency Communications (URLLC) and throughput of enhanced Mobile Broadband (eMBB) UEs, while traffic types are considered homogeneous [26].…”

An Innovative Machine-Learning-Based Scheduling Solution for Improving Live UHD Video Streaming Quality in Highly Dynamic Network Environments

“…An autonomous network resource management for QoS and QoE provisioning is proposed in [12] to predict the amount of network resources that needs to be allocated to cope with the traffic demands for live and on-demand dynamic adaptive streaming over HTTP. Machine learning is used to optimize the scheduling and resource allocation problems in 5G radio access networks focusing on different combinations of QoS objectives, such as: throughput, delay and packet loss in [18], packet loss and delay in [24], system throughput and user fairness in [25]. However, these ML-based scheduling solutions are designed for homogeneous traffic types only.…”

Wi-Fi Meets ML: A Survey on Improving IEEE 802.11 Performance with Machine Learning

“…access is a common feature for Wi-Fi and 5G, which was adopted by WLANs after it was successfully applied in the cellular domain. In the following papers ML is used to the address the several problems in OFDMA-based cellular networks: fair scheduling [149], [150], carrier frequency offset (CFO) estimation for uplink transmissions [151], [152], internetwork interference control [153], and resource allocation [154]. They implement RL [149], [150], [153], supervised deep learning [151], unsupervised deep learning [152], and a genetic learning algorithm [154] to support performance optimization.…”

“…In the following papers ML is used to the address the several problems in OFDMA-based cellular networks: fair scheduling [149], [150], carrier frequency offset (CFO) estimation for uplink transmissions [151], [152], internetwork interference control [153], and resource allocation [154]. They implement RL [149], [150], [153], supervised deep learning [151], unsupervised deep learning [152], and a genetic learning algorithm [154] to support performance optimization. We believe that these papers may provide interesting insights and guidelines for researchers working in the Wi-Fi domain.…”