DRL-based Path Planner and its Application in Real Quadrotor with LIDAR

Yang, Yongsheng; Hou, Zhengxin; Chen, Hongbo; Lu, Peng

doi:10.1007/s10846-023-01819-0

Cited by 5 publications

(7 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, a UAV simulation training and testing environment is built based on [20] to verify the proposed unified framework. Three sets of experiments are used to independently test the temporal characteristics, spatial characteristics and data denoising included in figure 2 and analyze the results to verify the effectiveness and rationality of proposed framework.…”

Section: Experiments and Discussionmentioning

confidence: 99%

“…References [29] and [30] both used the AirSim simulation platform to create highly realistic scenes to train agents, but the effort and cost increase as the degree of restoration of the actual scene increases. Different from the ideas in the appealed literature [20], trained the agent through dynamic training scenarios to enhance the UAV's reliability in many application scenarios. This method can reduce the influence of distribution mismatch problems at a lower cost.…”

Section: Relate Workmentioning

confidence: 99%

“…In addition, the training environment and application scenarios of UAVs usually have a serious distribution mismatch problem, resulting in agents trained in the training scenarios having poor adaptability in the application scenarios of UAV. The above problems make it difficult to use DRL for UAV autonomous navigation and obstacle avoidance [18][19][20]. Therefore, how to increase the autonomous navigation performance of UAVs from the training environment to the application environment is a work of great significance and challenge.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A framework for improving UAV decision of autonomous navigation from training to application migration under perceptual uncertainty

Sheng,

Liu,

et al. 2024

Meas. Sci. Technol.

View full text Add to dashboard Cite

Unmanned Aerial Vehicles (UAVs) autonomous navigation based on reinforcement learning (RL) usually requires training agents in simulation scenarios and then transferring the trained agents to application scenarios. However, due to serious distribution mismatch between the idealized simulation scenario and the application environment and the inevitable uncertainty perception problem of airborne sensors in complex scenarios, the navigation performance of UAV under migration applications is not ideal. This work fully analyzes the factors that affect UAV navigation performance, including algorithm performance, training strategy, and state awareness. Based on the analysis results, this article proposes a framework to improve the autonomous navigation performance of UAVs in the migration process from training to application, which consists of three parts: "scenario-perception-algorithm". In addition, this paper proposes improvement strategies for each part from the perspectives of spatial features, temporal features, and perceptual denoising. We combine the proposed framework with navigation algorithms to improve the navigation decision-making performance of UAVs in migration applications under uncertainty perception. Many simulation experiments demonstrate the effectiveness of the proposed framework and its robustness to uncertainty perception.

show abstract

Section: Experiments and Discussionmentioning

confidence: 99%

Section: Relate Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A framework for improving UAV decision of autonomous navigation from training to application migration under perceptual uncertainty

Sheng,

Liu,

et al. 2024

Meas. Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…During interactions, the agent receives immediate rewards, using them to evaluate its actions. In [38], Yang compared the SAC algorithm with deep deterministic policy gradients (DDPG) [39] and twin-delayed deep deterministic (TD3) [40] algorithms in the quadrotor obstacle avoidance task. Simulation results show that SAC outperforms DDPG and TD3 in terms of stability and performance.…”

Section: Soft Actor-criticmentioning

confidence: 99%

“…Note that in our approach we mainly consider kinematic aspects to control the motion of the UAV. Compared to [38], we incorporate the following point into the state. As mentioned earlier, sampling-based path planning algorithms are well suited for obtaining a predefined path that the DRL-based planner can follow.…”

Section: Network Structurementioning

confidence: 99%

Multi-UAV Path Planning and Following Based on Multi-Agent Reinforcement Learning

Zhao,

Yang,

Zhong

et al. 2024

Drones

Self Cite

View full text Add to dashboard Cite

Dedicated to meeting the growing demand for multi-agent collaboration in complex scenarios, this paper introduces a parameter-sharing off-policy multi-agent path planning and the following approach. Current multi-agent path planning predominantly relies on grid-based maps, whereas our proposed approach utilizes laser scan data as input, providing a closer simulation of real-world applications. In this approach, the unmanned aerial vehicle (UAV) uses the soft actor–critic (SAC) algorithm as a planner and trains its policy to converge. This policy enables end-to-end processing of laser scan data, guiding the UAV to avoid obstacles and reach the goal. At the same time, the planner incorporates paths generated by a sampling-based method as following points. The following points are continuously updated as the UAV progresses. Multi-UAV path planning tasks are facilitated, and policy convergence is accelerated through sharing experiences among agents. To address the challenge of UAVs that are initially stationary and overly cautious near the goal, a reward function is designed to encourage UAV movement. Additionally, a multi-UAV simulation environment is established to simulate real-world UAV scenarios to support training and validation of the proposed approach. The simulation results highlight the effectiveness of the presented approach in both the training process and task performance. The presented algorithm achieves an 80% success rate to guarantee that three UAVs reach the goal points.

show abstract