State Aware-Based Prioritized Experience Replay for Handover Decision in 5G Ultradense Networks

Abstract

The traditional handover decision methods depend on the handover threshold and measurement reports, which cannot efficiently resolve the frequent handover issue and ping-pong effect in 5G (5 generation) ultradense networks. To reduce the unnecessary handover and improve the QoS (quality of service), combine with the analysis of dwell time, we propose a state aware-based prioritized experience replay (SA-PER) handover decision method. First, the cell dwell time is computed by the geometrical analysis of real-time locations of mobile users in cellular networks. The constructed state aware sequence including SINR, load coefficient, and dwell time is normalized by max-min normalization method. Then, the handover decision problem in 5G ultradense networks is formalized as a discrete Markov decision process (MDP). The random sampling and small batch sampling affect the performance of deep reinforcement learning methods. We adopt the prioritized experience replay (PER) method to resolve the learning efficiency problems. The state space, action space, and reward functions are designed. The normalized state aware decision matrix inputs the DDQN (double deep Q-network) method. The competitive and collaborative relationships between vertical handover and horizontal handover in 5G ultradense networks are mainly discussed. And the high average network throughput and long average cell dwell time make sure of the communication quality for mobile users.

Advances and Challenges in Learning from Experience Replay

“…Appendix A presents the detailed search results, methodology, and applied criteria. We could verify that many works in the literature apply well-known RL methods to approach different classes of complex and exciting problems such as: (i) geographical routing-decision process to assign sensing tasks to mobile users, (ii) anomaly detection in smart environments, (iii) cellular-connected unmanned aerial vehicles network, (iv) nonlinearities and uncertainties of biochemical reactions in wastewater treatment process control, (v) robotic lever control, (vi) handover decision in 5G Ultradense Networks, and (vii) automation of software test (Zhang and Qiu, 2022;Tao and Hafid, 2020;Fährmann et al, 2022;Koroglu and Sen, 2022;Crowder et al, 2021;Li et al, 2022b;Wu et al, 2022;Remman and Lekkas, 2021;Rosenbauer et al, 2020). However, we are more interested in those works that propose changes or new methods using Experience Replay.…”

Research on Adaptive Handover Decision Algorithm in Next-Generation High-Speed Railway Scenario

“…Tere are various uncertainties in the running process of trains, which will bring potential risks to the handover of high-speed trains [21]. To solve this problem, a handover risk assessment model based on prospect theory is constructed, and the handover risk of trains of diferent handover positions is evaluated by using prospect theory [22].…”

“…Te nearness degree obtained by improving the CRITIC-TOPSIS joint handover decision method and the handover risk prospect value obtained by prospect theory is normalized and summed by equation (21), and the comprehensive utility values of trains in diferent high-speed rail scenarios are obtained, as shown in Figure 7.…”