Energy‐aware mobility for aerial networks: A reinforcement learning approach

Abstract

With recent advancements in aerial networks, aerial base stations (ABSs) have become a promising mobile network technology to enhance the coverage and capacity of the cellular networks. ABS deployment can assist cellular networks to support network infrastructure or minimize the disruptions caused by unexpected and temporary situations. However, with 3D ABS placement, the continuity of the service has increased the challenge of providing satisfactory Quality of Service (QoS). The limited battery capacity of ABSs and continuous movement of users result in frequent interruptions. Although aerial networks provide quick and effective coverage, ABS deployment is challenging due to the user mobility, increased interference, handover delay, and handover failure. In addition, once an ABS is deployed, an intelligent management must be applied. In this paper, we model user mobility pattern and formulate energyaware ABS deployment problem with a goal of minimizing energy consumption and handover delay. To this end, the contributions of this paper are threefold: (i) analysis of reinforcement learning (RL)-based state action reward state action (SARSA) algorithm to deploy ABSs with an energy consumption model, (ii) predicting the user next-place with a hidden Markov model (HMM), and (iii) managing the dynamic movement of ABSs with a handover procedure. Our model is validated by comprehensive simulation, and the results indicate superiority of the proposed model on deploying multiple ABSs to provide the communication coverage.