Deep Reinforcement Learning for UAV Routing in the Presence of Multiple Charging Stations

Fan, Mingfeng; Wu, Yaoxin; Liao, Tianjun; Cao, Zhiguang; Guo, Hongliang; Sartoretti, Guillaume; Wu, Guohua

doi:10.1109/tvt.2022.3232607

Cited by 13 publications

(4 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reference [21] introduced the state-of-the-art technologies for UAV-assisted maritime communications, discussed the physical layer, resource management, cloud/edge computing, and caching UAV-assisted solutions in maritime environments and grouped them according to their performance objectives. Reference [22] presents a deep reinforcement learning-based method based on the design of a multi-head heterogeneous attention mechanism to facilitate the learning of a policy that automatically and sequentially constructs the route, while taking energy consumption into account.…”

Section: Related Workmentioning

confidence: 99%

Reinforcement Learning-Based Resource Allocation and Energy Efficiency Optimization for a Space–Air–Ground-Integrated Network

Chen,

Zhou,

et al. 2024

Electronics

View full text Add to dashboard Cite

With the construction and development of the smart grid, the power business puts higher requirements on the communication capability of the network. In order to improve the energy efficiency of the space–air–ground-integrated power three-dimensional fusion communication network, we establish an optimization problem for joint air platform (AP) flight path selection, ground power facility (GPF) association, and power control. In solving the problem, we decompose the problem into two subproblems, one is the AP flight path selection subproblem and the other is the GPF association and power control subproblem. Firstly, based on the GPF distribution and throughput weights, we model the AP flight path selection subproblem as a Markov Decision Process (MDP) and propose a multi-agent iterative optimization algorithm based on the comprehensive judgment of GPF positions and workload. Secondly, we model the GPF association and power control subproblem as a multi-agent, time-varying K-armed bandit model and propose an algorithm based on multi-agent Temporal Difference (TD) learning. Then, by alternately iterating between the two subproblems, we propose a reinforcement learning (RL)-based joint optimization algorithm. Finally, the simulation results indicate that compared to the three baseline algorithms (random path, average transmit power, and random device association), the proposed algorithm improves an overall energy efficiency of the system of 16.23%, 86.29%, and 5.11% under various conditions (including different noise power levels, GPF bandwidth, and GPF quantities), respectively.

show abstract

Section: Related Workmentioning

confidence: 99%

Reinforcement Learning-Based Resource Allocation and Energy Efficiency Optimization for a Space–Air–Ground-Integrated Network

Chen,

Zhou,

et al. 2024

Electronics

View full text Add to dashboard Cite

show abstract

“…They split the optimization problem into customer clustering problem and routing components and applied an encoder-decoder framework with RL to resolve it. Fan et al [21] employed a multi-head attention mechanism coupled with a DRL policy to design routes for an energy-limited UAV, however they assumed a fixed set of recharging locations for the UAV.…”

Section: A Related Workmentioning

confidence: 99%

Deep Reinforcement Learning for Green UAV-Assisted Data Collection

Mondal

Mishra

Prasad

et al. 2023

ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Unmanned aerial vehicles (UAVs) are capable of surveying expansive areas, but their operational range is constrained by limited battery capacity. The deployment of mobile recharging stations using unmanned ground vehicles (UGVs) significantly extends the endurance and effectiveness of UAVs. However, optimizing the routes of both UAVs and UGVs, known as the UAV-UGV cooperative routing problem, poses substantial challenges, particularly with respect to the selection of recharging locations. Here in this paper, we leverage reinforcement learning (RL) for the purpose of identifying optimal recharging locations while employing constraint programming to determine cooperative routes for the UAV and UGV. Our proposed framework is then benchmarked against a baseline solution that employs Genetic Algorithms (GA) to select rendezvous points. Our findings reveal that RL surpasses GA in terms of reducing overall mission time, minimizing UAV-UGV idle time, and mitigating energy consumption for both the UAV and UGV. These results underscore the efficacy of incorporating heuristics to assist RL, a method we refer to as heuristics-assisted RL, in generating high-quality solutions for intricate routing problems.

show abstract

“…The control of reflection properties by RIS opens up possibilities for optimizing signal strength and quality at the receiver. An optimal phase shift in RIS contributes to maximizing the number of served devices, improving the SNR, increasing the network sum rate, mitigating signal propagation impairments, and expanding the coverage and capacity in hybrid scenarios [10], [49], [50]. Therefore, optimizing the controllable phase shift towards the receiver becomes crucial for achieving enhanced performance.…”

Section: Optimizing Reflection Properties: a Paradigm Shift In Wirele...mentioning

confidence: 99%

Joint Beamforming Optimization Design and Performance Evaluation of RIS-Aided Wireless Networks: A Comprehensive State-of-the-Art Review

Ibrahim,

Mahmud,

Salleh

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Reconfigurable intelligent surface (RIS) has emerged as a promising technology for supporting sixth-generation (6G) wireless communication systems. RIS intelligently manages signal reflection properties to enhance signal strength and quality at the receiver. RIS devices are inherently passive, requiring minimal power to perform signal manipulation. Through signal path optimization, interference and transmit power reduction, RIS advances energy efficiency. This aligns with the eco-friendly goals of 6G. However, efficient optimization in RIS-aided networks plays a crucial role in fully unlocking the potential of RIS. Additionally, performance evaluation of RIS-aided systems contributes to understanding optimal configurations and design guidelines for real-world RIS deployments. Despite the contributions from performance evaluation and optimization, RIS confronts various design constraints that demand innovative approaches to optimization and evaluation of their performances. Implementing RIS poses challenges in optimizing configurations for dynamic environments, handling computational complexity in large-scale systems, and adapting to changing wireless conditions. This work provides a comprehensive and in-depth survey of the state-of-the-art in joint optimization designs and performance evaluation for RIS-aided wireless systems. Furthermore, we identify areas that require further investigation to facilitate future research in addressing the existing gaps in knowledge. Finally, the survey discusses promising future research directions that include exploring optimization techniques ensuring global optima with computational efficiency, investigating analytical approaches for accurate network performance assessment under diverse conditions, advancing multiobjective joint optimization, developing application-specific integrated performance metrics, examining the joint impact of relevant RIS design parameters, investigating adaptive quantization schemes, and exploring reliable channel state estimation techniques that are resilient against dynamic environments, noise, interference, and delays.

show abstract

Deep Reinforcement Learning for UAV Routing in the Presence of Multiple Charging Stations

Cited by 13 publications

References 42 publications

Reinforcement Learning-Based Resource Allocation and Energy Efficiency Optimization for a Space–Air–Ground-Integrated Network

Reinforcement Learning-Based Resource Allocation and Energy Efficiency Optimization for a Space–Air–Ground-Integrated Network

Deep Reinforcement Learning for Green UAV-Assisted Data Collection

Joint Beamforming Optimization Design and Performance Evaluation of RIS-Aided Wireless Networks: A Comprehensive State-of-the-Art Review

Contact Info

Product

Resources

About