Online Radio Pattern Optimization Based on Dual Reinforcement-Learning Approach for 5G URLLC Networks

Abstract

The fifth generation (5G) radio access technology is designed to support highly delaysensitive applications, i.e., ultra-reliable and low-latency communications (URLLC). For dynamic time division duplex (TDD) systems, the real-time optimization of the radio pattern selection becomes of a vital significance in achieving decent URLLC outage latency. In this study, a dual reinforcement machine learning (RML) approach is developed for online pattern optimization in 5G new radio TDD deployments. The proposed solution seeks to minimizing the maximum URLLC tail latency, i.e., min-max problem, by introducing nested RML instances. The directional and real-time traffic statistics are monitored and given to the primary RML layer to estimate the sufficient number of downlink (DL) and uplink (UL) symbols across the upcoming radio pattern. The secondary RML sub-networks determine the DL and UL symbol structure which best minimizes the URLLC outage latency. The proposed solution is evaluated by extensive and highly-detailed system level simulations, where our results demonstrate a considerable URLLC outage latency improvement with the proposed scheme, compared to the state-of-the-art dynamic-TDD proposals.

Analysis of Outage Latency and Throughput Performance in Industrial Factory 5G TDD Deployments

“…Finally, we investigate the potential of the RL-based TDD frame selection solution [19] compared to the non-RL based TDD frame selection schemes, i.e., reactive TDD, where only the URLLC traffic is considered. Fig.…”

“…In line with the system modeling assumptions in [19], we explicitly consider the major functionalists of the 5G NR PHY and MAC layers. In the DL direction, arriving packets are first processed, and are buffered towards the first available DL TTI of the current TDD radio frame.…”

“…Furthermore, a semi-static TDD adaptation algorithm [18] is proposed to avoid the network CLI while offering a semi-static dynamicity of the network TDD radio frame to traffic demands. Finally, a reinforcementlearning (RL) based TDD frame optimization scheme [19] has been proposed to autonomously optimize the BS-specific TDD frame selection in a distributed manner, where the achievable learning gain offers a considerable URLLC performance improvement compared to reactive TDD adaptation schemes. Therein, two learning instances have been defined.…”

Power Saving Techniques in 3GPP 5G New Radio: A Comprehensive Latency and Reliability Analysis

“…Both downlink and uplink directions are separately considered. The URLLC-alike FT3 traffic is considered with small and sporadic payload transmissions, where the offered traffic load per cell is calculated in line with [9].…”

“…On another side, and away from the power efficiency targets, the feasibility of the URLLC latency and reliability requirements is investigated for macro deployments with frequency division duplex (FDD) [6], time division duplex (TDD) [7], and for industrial factory TDD deployment [2], respectively. Furthermore, smarter scheduling algorithms [8,9] are vital to achieve the challenging URLLC targets, particularly in coexistence deployments with various quality of service (QoS) classes on the same spectrum. However, there is lack in the state-of-the-art literature on the performance cost, i.e., radio latency and reliability, paid in order to fulfill a power efficient operation at the device side.…”