Efficient 3D aerial base station placement considering users mobility by reinforcement learning

Abstract

This paper considers an aerial base station (aerial-BS) assisted terrestrial network where user mobility is taken into account. User movement changes the network dynamically which may result in performance loss. To avoid this loss, guarantee a minimum quality-of-service (QoS) and possibly increase the QoS, we add an aerial-BS to the network. For fair comparison between the conventional terrestrial network and the aerial-BS assisted one, we keep the total number of BSs identical in both networks. Obtaining the best performance in such networks highly depends on the optimal placement of the aerial-BS. To this end, an algorithm which can rely on general and realistic assumptions and can decide where to go based on the past experiences is required. The proposed approach for this goal is based on a discounted reward reinforcement learning which is known as Q-learning. Simulation results show this method provides an effective placement strategy which increases the QoS of wireless networks when it is needed and promises to find the optimum position of the aerial-BS in discrete environments.

On-demand deployment of multiple aerial base stations for traffic offloading and network recovery

“…Further extensions on that work included deploying a single UAV to maximize the coverage of a set of users with different quality of service (QoS) requirements [25]. In [26], the authors utilized a Q-learning technique to find a 3D position of one aerial base station in order to maximize the aggregate throughput of a terrestrial network while accounting for user mobility. In addition, the authors in [27] derive the optimal altitude of a single UAV to achieve maximum downlink ground coverage and minimum transmit power consumption; this latter work then investigates the optimal deployment of two UAVs to maximize coverage and determine the altitude of the UAVs as well as the distance separating them.…”

A Novel Model Representation Framework for Cooperative Intelligent Transport Systems

“…On the other hand, in V2I systems it can be used to provide communication channel between the vehicle and the traffic signal systems. WIFI and mobile (cellular) network technology such as GSM (Global System for Mobile communication) can provide a capable, secured (Ghanavi et al, 2018) and reliable communication (Gopalakrishnan et al, 2018) solutions for V2X applications.…”

Comparison of Optimization Methods for Aerial Base Station Placement with Users Mobility

“…In a Q-learning algorithm, the agent considers a class of states S = {s 1 , s 2 , ..., s n }, a class of actions A = {a 1 , a 2 , ..., a m }, and a knowledge matrix Q. In each state, the learning agent performs an action a t which triggers a state transition [3]. Then, the agent calculates the reward in the new state, and the matrix Q is updated.…”

“…an optimization with incomplete knowledge of the future, able to follow the changes in the scenario where the optimization is performed. In both [3] and [4], a reinforcement learning technique, Q-learning, is used to perform the online optimization and maximize the sum of the rates of the users. In [3], it is optimized the 3D position of an aerial base station which complements a terrestrial network, whereas, in [4], the focus is on the deployment of multiple aerial base stations in an urban area.…”