A Stealth–Distance Dynamic Weight Deep Q-Network Algorithm for Three-Dimensional Path Planning of Unmanned Aerial Helicopter

Abstract

Unmanned aerial helicopters (UAHs) have been widely used recently for reconnaissance operations and other risky missions. Meanwhile, the threats to UAHs have been becoming more and more serious, mainly from radar and flights. It is essential for a UAH to select a safe flight path, as well as proper flying attitudes, to evade detection operations, and the stealth abilities of the UAH can be helpful for this. In this paper, a stealth–distance dynamic weight Deep Q-Network (SDDW-DQN) algorithm is proposed for path planning in a UAH. Additionally, the dynamic weight is applied in the reward function, which can reflect the priorities of target distance and stealth in different flight states. For the path-planning simulation, the dynamic model of UAHs and the guidance model of flight are put forward, and the stealth model of UAHs, including the radar cross-section (RCS) and the infrared radiation (IR) intensity of UAHs, is established. The simulation results show that the SDDW-DQN algorithm can be helpful in the evasion by UAHs of radar detection and flight operations, and the dynamic weight can contribute to better path-planning results.

Stealth Aircraft Penetration Trajectory Planning in 3D Complex Dynamic Based on Radar Valley Radius and Turning Maneuver

“…Alpdemir [20] proposed a deep reinforcement learning solution for the trajectory planning problem of tactical unmanned aerial vehicles under the threat of radar tracking, integrating a Markov Decision Process with a variant of Deep Q-Networks and prioritized experience replay, along with Learning from Demonstrations (LfD). Wang Z [21] and colleagues introduced a Concealment-Distance Dynamic Weight Deep Q-Network algorithm for the three-dimensional trajectory planning of unmanned helicopters, which considers radar and infrared detection threats and optimizes trajectory planning outcomes through a dynamic weighting reward function. Through comparative analysis, Hameed et al [22] studied the application of deep reinforcement learning algorithms in aircraft avoidance or the minimization of radar detection and tracking, finding that the Proximal Policy Optimization (PPO) algorithm generally performs better.…”