LB‐DDQN for Handover Decision in Satellite‐Terrestrial Integrated Networks

Abstract

The frequent handover and handover failure problems obviously degrade the QoS of mobile users in the terrestrial segment (e.g., cellular networks) of satellite-terrestrial integrated networks (STINs). And the traditional handover decision methods rely on the historical data and produce the training cost. To solve these problems, the deep reinforcement learning- (DRL-) based handover decision methods are used in the handover management. In the existing DQN-based handover decision method, the overestimates of DQN method continue. Moreover, the current handover decision methods adopt the greedy strategy which lead to the load imbalance problem in base stations. Considering the handover decision and load imbalance problems, we proposed a load balancing-based double deep Q-network (LB-DDQN) method for handover decision. In the proposed load balancing strategy, we define a load coefficient to express the conditions of loading in each base station. The supplementary load balancing evaluation function evaluates the performance of this load balancing strategy. As the selected basic method, the DDQN method adopts the target Q-network and main Q-network to deal with the overestimate problem of the DQN method. Different from joint optimization, we input the load reward into the designed reward function. And the load coefficient becomes one handover decision factor. In our research, the handover decision and load imbalance problems are solved effectively and jointly. The experimental results show that the proposed LB-DDQN handover decision method obtains good performance in the handover decision. Moreover, the access of mobile users becomes more balancing and the throughput of network is also increased.

State Aware-Based Prioritized Experience Replay for Handover Decision in 5G Ultradense Networks

“…The Q-learning method is not suitable for the decision problem with high dimension state space. The DQN method replaces the Q table with DNN to describe the action value function, which is used to solve the decision problem with high dimension state space [32].…”

“…The global information is used in the training process of decentralized policy used in UE. In [32], Wu et al proposed a load balancing-based double deep Q-network (LB-DDQN) method for handover decision. In the proposed load balancing strategy, a load coefficient is defined to express the conditions of loading in each base station.…”

A Fuzzy-Logic-Based Load Balancing Scheme for a Satellite–Terrestrial Integrated Network

“…The authors of [18] analyzed the transmission characteristics of terrestrial and back-haul links to propose a greedy-based user association algorithm and a matching algorithm with user grouping for balancing the load by performing multiple iterations between users and cells. In [19], the authors noted that the current methods adopt the greedy strategy, which leads to the load imbalance problem in cells. Thus, they defined a load coefficient and added it to the reward function to make handover decisions while balancing loads.…”