Joint Optimization of User Association and Resource Allocation in HetNets: A parameterized-DRL Approach

Abstract

Facing the problem of resources utilization in of multiple wireless communication systems with multiple coverage, convergence of heterogeneous network (HetNets) can reduce the burden and decrease deployment costs of a single communication network. Due to the non-convexity of joint optimization and the difficulty to obtain the feasible solution of the mixed operation space (i.e. discrete operation versus continuous operation) of HetNets, so it is a formidable challenge to achieve global optimization of energy efficiency (EE) and spectral efficiency (SE) simultaneously when facing user association (discrete) together with power allocation (continuous). Unlike the method of deep reinforcement learning (DRL) by discretizing continuous space directly, we proposed a novel parameterized-DRL that maximizes the performance of joint EE-SE while ensuring the quality of service (QoS) of downlink user devices (UEs) in HetNets. In addition, to solve the computationally intensive problem in the state-action space, an algorithm of parameterized-experience-replay dueling double DQN with multi-agent priority (P-MAPD3QN) is introduced to obtain an almost optimal QoS. Simulation results show that this algorithm improves the effectiveness of the system by 4.9% over traditional D3QN algorithm in terms of system capacity and 13.1% in terms of joint EE-SE performance.