UAV Path Planning Based on Multi-Layer Reinforcement Learning Technique

Abstract

Unmanned aerial vehicles (UAVs) have been widely used in many applications due to its small size, swift mobility and low cost. Therefore, the study of guidance, navigation and control (GNC) system of UAV has becoming a popular research direction. Path planning plays an important role in the GNC system. In this paper, a multi-layer path planning algorithm based on reinforcement learning (RL) technique is proposed. Compared to the classic Q-learning, the proposed multi-layer algorithm has a distinct advantage that it collects both global and local information which greatly improves overall performance. The proposed RL algorithm has two layers, the higher layer deals with the local information, which could be considered as a short-term strategy. The lower layer deals with the global information, which could be considered as a long-term strategy. Both the higher layer and lower layer are working in harmony to plan a collision-free path. B-spline curve approach is applied for on-line path smoothing. Simulation results in different scenarios prove the effectiveness of multi-layer Q-learning algorithm.

Design and Analysis of a Novel Concept-Based Unmanned Aerial Vehicle with Ground Traversing Capability

“…Yao and Zhao [20] suggested the model predictive control algorithm to search optimal or sub-optimal collision-free trajectories for a UAV in the midst of dynamic obstacle conditions. However, Besada-Portas et al [18]; Cui and Wang [19]; and Yao and Zhao [20] have shown the software-based computer simulation results of UAVs in their studies. Heidari and Saska [9] developed a heuristic approach-based open-loop control system to select optimal values of the dynamic control parameters such as thrust force and torque of a quadcopter for trajectory optimisation.…”

“…Comparative performance analysis of different multiobjective evolutionary algorithms as a path planner of UAV has been studied by Besada-Portas et al [18]). Cui and Wang [19] implemented the reinforcement learning algorithm to design local and global path planners for a UAV among static and moving obstacles. Yao and Zhao [20] suggested the model predictive control algorithm to search optimal or sub-optimal collision-free trajectories for a UAV in the midst of dynamic obstacle conditions.…”

Mobile Robot Path Planning Using a QAPF Learning Algorithm for Known and Unknown Environments

“…In the literature, there are several proposals to address path planning problems using reinforcement learning [11], [12]. Some proposals based on reinforcement learning have been combined with other techniques to improve performance.…”

Autonomous Unmanned Aerial Vehicle navigation using Reinforcement Learning: A systematic review

“…Some maps oversimplify the environment representation: the map is divided into a grid with equally-sized smaller cells that store information about the environment [68,73,93]. Others oversimplify the environment's structure by simplifying objects representation or by using 1D/2D to represent the environment [34,36,39,41,47,55,65,67,74,86,87] .…”

“…• Step 1 -Define State Type: When assessing an RL task, it is essential to comprehend the state that can be obtained from the surrounding environment. For instance, some navigation tasks simplify the environment's states using gridcell representations [68,73,93], where the agent has a limited and predetermined set of states, whereas in other tasks, the environment can have unlimited states [34,38,40]. Therefore, this steps involves a decision between limited vs. unlimited states.…”