Deep Reinforcement Learning Approach for Joint Trajectory Design in Multi-UAV IoT Networks

Xu, Shun; Zhang, Xiangyu; Li, Chunguo; Wang, Dongming; Yang, Luxi

doi:10.1109/tvt.2022.3144277

Cited by 29 publications

(8 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their work is potentially applicable for smart farms and factories. Xu et al implemented k-means and deep reinforcement learning algorithms to optimize multi-UAV trajectory for uplink data collection in IoT networks [274]. The algorithm aimed to minimize data collection time while considering criteria such as maximum speed, maximum acceleration, collision avoidance, and UAV communication interference with promising results.…”

Section: Internet Of Things (Iot)mentioning

confidence: 99%

Unmanned Aerial Vehicle Communications for Civil Applications: A Review

et al. 2022

View full text Add to dashboard Cite

The use of drones, formally known as unmanned aerial vehicles (UAVs), has significantly increased across a variety of applications over the past few years. This is due to the rapid advancement towards the design and production of inexpensive and dependable UAVs and the growing request for the utilization of such platforms particularly in civil applications. With their intrinsic attributes such as high mobility, rapid deployment and flexible altitude, UAVs have the potential to be utilized in many wireless system applications. On the one hand, UAVs are able to operate as flying mobile terminals within wireless/cellular networks to support a variety of missions such as goods delivery, search and rescue, precision agriculture monitoring, and remote sensing. On the other hand, UAVs can be utilized as aerial base stations to increase wireless communication coverage, reliability, and the capacity of wireless systems without additional investment in wireless systems infrastructure. The aim of this article is to review the current applications of UAVs for civil and commercial purposes. The focus of this paper is on the challenges and communication requirements associated with UAV-based communication systems. This article initially classifies UAVs in terms of various parameters, some of which can impact UAVs' communication performance. It then provides an overview of aerial networking and investigates UAVs routing protocols specifically, which are considered as one of the challenges in UAV communication. This article later investigates the use of UAV networks in a variety of civil applications and considers many challenges and communication demands of these applications. Subsequently, different types of simulation platforms are investigated from a communication and networking viewpoint. Finally, it identifies areas of future research.

show abstract

Section: Internet Of Things (Iot)mentioning

confidence: 99%

Unmanned Aerial Vehicle Communications for Civil Applications: A Review

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Given the recent advances in deep reinforcement learning (DRL), the authors of [4][5][6][7][8] proposed techniques based on DRL for resolving the above challenges for AoI-optimal transmission policies. Li et al [4] considered discretized trajectories and harnessed deep Q-network (DQN) to design their control policies.…”

Section: Introductionmentioning

confidence: 99%

“…To learn actions involving both continuous and discrete variables, Hu et al [7] combines DQN and DDPG to design UAVs trajectories for AoI minimization. To tackle multi-agent trajectory planning problems, Xu et al [8] utilized an independent agent based Q-Learning method for minimizing the mission completion time of multi-UAVaided data collection. However, the solutions in [4][5][6][7][8] learn the policy for each agent independently while treating other simultaneously-learning agents as part of the environment, which results in the non-stationary problems.…”

Section: Introductionmentioning

confidence: 99%

Learning-Aided UAV-Cooperation Reduces the Age-of-Information in Wireless Networks

Chen,

Liu,

Zhang

et al. 2024

IEEE Commun. Lett.

View full text Add to dashboard Cite

Unmanned aerial vehicles (UAVs) can enhance data collection for ground sensing nodes (SNs). Given the modest battery capacity of UAVs and the limited communication range of SNs, it is crucial to conceive efficient trajectory coordination for UAVs. However, existing studies simply decouple the joint trajectory planning policy of multiple UAVs into independent local policies, preventing their cooperation and hence limits the performance. Inspired by the observation that sharing messages among agents can promote their cooperation, we investigate the communication-assisted decentralized trajectory planning policy of multi-UAV wireless networks. Our goal is to minimize the overall energy consumption of UAVs and the average age of information of all SNs. To harness the encoded messages for learning a sophisticated policy, we conceive a communicationassisted distributed training and execution framework, and propose a communication-aided decentralized trajectory control algorithm. Our simulation results show that the proposed algorithm substantially outperforms the state-of-the-art deep reinforcement learning based methods, at a modest communication overhead.

show abstract

“…It is a promising way to apply RL algorithms to solve the UAV maneuver decision-making problem (Hu et al, 2022). Regarding long-horizon tasks, the RL agent depends on dense rewards to train and construct long-term decision sequences (Xu et al, 2022;Nagpal et al, 2020). However, reward signals are highly sparse in the UAV maneuver decision-making problem due to the lack of expert experience (Sun et al, 2021).…”

Section: Introductionmentioning

confidence: 99%