Reinforcement Learning Aided UAV Base Station Location Optimization for Rate Maximization

Abstract

The application of unmanned aerial vehicles (UAV) as base station (BS) is gaining popularity. In this paper, we consider maximization of the overall data rate by intelligent deployment of UAV BS in the downlink of a cellular system. We investigate a reinforcement learning (RL)-aided approach to optimize the position of flying BSs mounted on board UAVs to support a macro BS (MBS). We propose an algorithm to avoid collision between multiple UAVs undergoing exploratory movements and to restrict UAV BSs movement within a predefined area. Q-learning technique is used to optimize UAV BS position, where the reward is equal to sum of user equipment (UE) data rates. We consider a framework where the UAV BSs carry out exploratory movements in the beginning and exploitary movements in later stages to maximize the overall data rate. Our results show that a cellular system with three UAV BSs and one MBS serving 72 UE reaches 69.2% of the best possible data rate, which is identified by brute force search. Finally, the RL algorithm is compared with a K-means algorithm to study the need of accurate UE locations. Our results show that the RL algorithm outperforms the K-means clustering algorithm when the measure of imperfection is higher. The proposed algorithm can be made use of by a practical MBS–UAV BSs–UEs system to provide protection to UAV BSs while maximizing data rate.

Aprendizado Federado e Deep Q-Network habilitando VANTs como Infraestrutura em Redes 6G

“…No aprendizado por reforc ¸o (Reinforcement Learning -RL), os modelos são treinados para elaborar uma sequência de decisões. Neste paradigma, não existem dados de treinamento, pois os algoritmos se baseiam em um modelo de recompensas e punic ¸ões à medida que o agente interage com o ambiente [P G and Magarini 2021] .…”

“…Portanto, espera-se que os VANTs fornec ¸am servic ¸os de comunicac ¸ão para novas aplicac ¸ões inteligentes orientadas por dados [Brik et al 2020]. Nesse contexto, as abordagens assistidas por Aprendizado de Máquina (Machine Learning -ML) tem sido utilizadas extensivamente para fins de otimizac ¸ão e automac ¸ão [P G and Magarini 2021].…”

Prioritized User Association for Sum-Rate Maximization in UAV-Assisted Emergency Communication: A Reinforcement Learning Approach

“…The deployment of a UAV by optimizing its trajectory to maximize the mean opinion score using a deep Q-learning method was presented in [23]. Similarly, the data rate maximization of an ABS-assisted downlink cellular system using RL was presented in [24]. Here, the Q-learning technique was used to optimize the ABS location, where simulation results revealed that RL performed better than a k-means algorithm to find the optimal ABS positions.…”

The UAV Path Coverage Algorithm Based on the Greedy Strategy and Ant Colony Optimization

“…With learning-based methods are widely used [28]. In [29][30][31], methods such as YOLOv4-CSP, Q-learning and Deep Reinforcement Learning are used to improve the speed and accuracy of the algorithms. However, the algorithm cannot resolve whether the flight path follows the original global path planning method or performs local path planning first and then replans.…”