Path Planning Research of a UAV Base Station Searching for Disaster Victims’ Location Information Based on Deep Reinforcement Learning

Abstract

Aiming at the path planning problem of unmanned aerial vehicle (UAV) base stations when performing search tasks, this paper proposes a Double DQN-state splitting Q network (DDQN-SSQN) algorithm that combines state splitting and optimal state to complete the optimal path planning of UAV based on the Deep Reinforcement Learning DDQN algorithm. The method stores multidimensional state information in categories and uses targeted training to obtain optimal path information. The method also references the received signal strength indicator (RSSI) to influence the reward received by the agent, and in this way reduces the decision difficulty of the UAV. In order to simulate the scenarios of UAVs in real work, this paper uses the Open AI Gym simulation platform to construct a mission system model. The simulation results show that the proposed scheme can plan the optimal path faster than other traditional algorithmic schemes and has a greater advantage in the stability and convergence speed of the algorithm.

A Stealth–Distance Dynamic Weight Deep Q-Network Algorithm for Three-Dimensional Path Planning of Unmanned Aerial Helicopter

“…In Equation (25), ρ represents the amplitude value of the summation of the infrared noise and the infrared signal of the UAH, and a represents the amplitude of the infrared signal of the UAH.…”

“…As one of the DRL algorithms, the Deep Q-Network (DQN) algorithm is a method to approximate the Q-learning function through a neural network. DQN methods have been increasingly applied in the field of path planning, and several brilliant algorithms based on it have been put forward [22][23][24][25]. Yin Cheng et al [26] have developed a concise DRL obstacle-avoidance algorithm that designed a comprehensive reward function for behaviors such as obstacle avoidance, target approach, speed correction, and attitude correction in dynamic environments, using the deep Q-network (DQN) architecture, to overcome the usability issue caused by the complicated control law in the traditional analytic approach.…”

Active robotic search for victims using ensemble deep learning techniques

“…Different solutions have been proposed, whether for unmanned aerial vehicles (UAVs) [12,13], submarines [14] or terrestrial robots [15]. However, in these existing approaches, there is a common limitation: they either do not incorporate visual information from the environment when planning their route [12,16], or if they do, they do so using deep learning techniques [17]. This means that active search-concept defined in [18] as 'on a large-scale environment [.…”

Energy-Aware Hierarchical Reinforcement Learning Based on the Predictive Energy Consumption Algorithm for Search and Rescue Aerial Robots in Unknown Environments

“…Inspired by human thinking on solving complex problems, HRL not only breaks down the problem into sub-problems that are easier to handle but has the ability to train multiple policies that are connected at different levels of temporal abstraction. HRL offers a structured approach for tasks involving multiple objectives, by segmenting decision-making into different layers [24]. Its application in aerial robot navigation has included coordinating multi-objective missions, exemplified in recent studies where HRL has been employed to optimize task allocation and path planning [25,26].…”