Reinforcement learning-based hybrid spectrum resource allocation scheme for the high load of URLLC services

Abstract

Ultra-reliable and low-latency communication (URLLC) in mobile networks is still one of the core solutions that require thorough research in 5G and beyond. With the vigorous development of various emerging URLLC technologies, resource shortages will soon occur even in mmWave cells with rich spectrum resources. As a result of the large radio resource space of mmWave, traditional real-time resource scheduling decisions can cause serious delays. Consequently, we investigate a delay minimization problem with the spectrum and power constraints in the mmWave hybrid access network. To reduce the delay caused by high load and radio resource shortage, a hybrid spectrum and power resource allocation scheme based on reinforcement learning (RL) is proposed. We compress the state space and the action space by temporarily dumping and decomposing the action. The multipath deep neural network and policy gradient method are used, respectively, as the approximater and update method of the parameterized policy. The experimental results reveal that the RL-based hybrid spectrum and the power resource allocation scheme eventually converged after a limited number of iterative learnings. Compared with other schemes, the RL-based scheme can effectively guarantee the URLLC delay constraint when the load does not exceed 130%.

The Frontiers of Deep Reinforcement Learning for Resource Management in Future Wireless HetNets: Techniques, Challenges, and Research Directions

“…Another Hybrid access networks are a special architecture for broadband access networks where different types of access networks are integrated to improve bandwidth. Huang et al [186] propose a single-agent DQN model to address the problem of delay minimization via joint spectrum and power resource allocation in mmWave mobile hybrid access network. The agent is located in the roadside BS, whose action space is discrete, corresponding to allocating spectrum and power resources for data.…”

Refiner GAN Algorithmically Enabled Deep-RL for Guaranteed Traffic Packets in Real-Time URLLC B5G Communication Systems

“…The portion of all RB 𝑧 𝑖𝑗 𝑡 is required for serving URLLC traffic overlaps at TTI. The reality of chance constraint in (5a) is still computationally expensive, and the combinatorial variable in ( 5) has difficulty reaching a globally optimal solution [3], [5], [22], [28]…”

“…From (7f), the value in every time slot 𝑡 shows more complexity, making it difficult to improve the cellular networks. The long order of time complexity and difficulty handling more active space in URLLC [5], [28]. The deep-RL algorithm must be able to address more action space in real-time.…”

“…The constraint (10a) guarantees that every UE can be performed with a required rate 𝑅 𝑖 𝑡 ≥ 𝒞 𝑖 𝑡 , and minimization of BS power in (10), which can be solved with constraint (10a) a form of inequality constraint 𝑅 𝑖 𝑡 ≥ 𝒞 𝑖 𝑡 , will be fulfilled in the form of equivalence [30]. The convex optimization problem in ( 10)-(10b) can be solved by the augmented Lagrangian algorithm [28]. To reduce the state space dimensions, the rate 𝑅 𝑖 𝑡 for every UE is defined by collecting the desired rate 𝒞 𝑖 𝑡 for every UE.…”

Deep Reinforcement Learning for Radio Resource Allocation and Management in Next Generation Heterogeneous Wireless Networks: A Survey

“…Huang et al [170] propose a single-agent DQN model to address the problem of delay minimization via joint spectrum and power resource allocation in mmWave mobile hybrid access network. The agent is located in the roadside BS, whose action space is discrete, corresponding to allocating spectrum and power resources for data.…”