Multi-Agent Deep Reinforcement Learning Assisted Pre-connect Handover Management

Abstract

Handover is an essential and significant component of mobility management in cellular networks. Handover management is more challenging for Fifth Generation (5G) networks because of ultra-reliable low latency communications (URLLC) requirements. This thesis proposes a make-before-break (MBB) adopted handover mechanism for user equipment (UE), namely, pre-connect handover (PHO). PHO aims at providing a seamless and reliable handover technique in 5G networks. PHO utilizes the Deep Q-Networks (DQN) algorithm to facilitate the sequential decision-making problem of the target base station (T-BS) selection based on the reference signal received quality (RSRQ) values and RSRQ change rates of all the candidate base stations (BSs). Furthermore, a multi-agent deep reinforcement learning (MADRL) solution is tailored to extend the DQN-assisted UEassociated PHO solution for modeling of the multi-UE environment. All the autonomous agents learn the action policy by interacting with the environment in a distributed manner. The feasibility of the PHO mechanism has been validated extensively via Network Simulator 3 (NS-3) and NS3-Gym. The performance of the DQN and MADRL-assisted PHO management solutions have been evaluated by considering various configurations.The experimental results demonstrated that the proposed PHO is not only achievable, but also that the DQN-assisted PHO technique can productively accomplish the optimal BS selection to maximize the success rate of PHO. Moreover, the MADRL-assisted PHO management solution can also be conducted and effectively applied to a realistic multi-UE environment where each UE is modeled with an agent.

Research on Multi-agent Sparse Reward Problem

“…Since the multi-agent system was proposed in the 1970s, it plays an important role in all walks of life. Based on the multi-agent system, the researchers introduced multi-agent reinforcement learning [3]. One of the main challenges facing RL algorithms is the sparse reward problem.…”