A UAV Navigation Approach Based on Deep Reinforcement Learning in Large Cluttered 3D Environments

Xue, Yuntao; Chen, Weisheng

doi:10.1109/tvt.2022.3218855

Cited by 16 publications

(6 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent research, feature-transfer-based DRL methods have demonstrated enhanced perception efficiency and superior navigation performance by using clearer and more informative inputs, such as visual labels or geometric data, for the UAV's state. This is shown in studies [31] and [32], where UAVs can directly determine the relative pose between the target and itself, as well as the collision distance with surrounding obstacles, using pure geometric data. However, precise and ideal geometric data pose considerable challenges for real-world applications, exhibiting a significant reality gap due to the presence of noises and imperfections.…”

Section: B Feature-transfer-based Methods For Agent Navigationmentioning

confidence: 99%

Semantic-Driven Autonomous Visual Navigation for Unmanned Aerial Vehicles

Yue,

Xin,

Zhang

et al. 2024

IEEE Trans. Ind. Electron.

View full text Add to dashboard Cite

Aiming at the autonomous navigation of unmanned aerial vehicles (UAVs) in complex and unknown environments, this article combines transfer reinforcement learning theory with an end-to-end mapless visual navigation method and proposes a semantic-driven autonomous navigation method for UAVs. First, we propose a reality reconstruction motion decision model, which consists of two parts: reality reconstruction and motion decision. The reality reconstruction part encodes the environment state into a unified feature space by combining semantic information and raw visual perception to generate a more efficient state description. In the motion decision part, we propose a motion decision network for navigation based on temporal attention and spatial attention mechanisms, which plans its own spatial perception according to key features and important memories to generate highly universal strong temporal navigation motion decisions. Subsequently, the action module generates and executes the optimal actions that the UAV can perform based on the decision results. The simulation results show that the UAV can autonomously explore and actively acquire navigation target position and complete collision-free autonomous navigation from any position to the navigation target in the absence of global information and unknown target positions. The real-world experimental testing results further demonstrate that this method improves the utilization efficiency of perception resources for UAVs, solves the reality gap problem, and has strong transferability. It can be directly transferred to challenging scenarios such as dynamic targets, multiple obstacles, and changes in target and obstacle types without the need for retraining. Overall, it is a highly autonomous and adaptive navigation method.

show abstract

Section: B Feature-transfer-based Methods For Agent Navigationmentioning

confidence: 99%

Semantic-Driven Autonomous Visual Navigation for Unmanned Aerial Vehicles

Yue,

Xin,

Zhang

et al. 2024

IEEE Trans. Ind. Electron.

View full text Add to dashboard Cite

show abstract

“…Long Trajectory Maze Dynamic Environment [18,19] [ [20][21][22][23][24] [27, 29,35] [ 28,38], [30,33,36,37] [ 41,42,50] this work…”

Section: Efficientmentioning

confidence: 99%

“…As a result, in recent years, various DRL-based obstacle-avoidance solutions [27][28][29][30][31][32][33][34][35][36][37][38] have gotten a lot of attention due to their real-time, kinodynamic, and energy-efficient features. DRL is a computational approach for learning how to map states to actions to obtain the optimized policy [39].…”

Section: Introductionmentioning

confidence: 99%

“…Because of its limited field-of-view sensing, it immediately discovers that the surroundings are without obstacles and then goes back, which eventually leads to the UAV hovering around in the corner. Based on the results of the state-of-art [38], although the DRL network is modified to train a policy for UAVs to escape from maze-like obstacles, there is still a 10% failure rate when the trajectory grows longer.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Hybrid Human-in-the-Loop Deep Reinforcement Learning Method for UAV Motion Planning for Long Trajectories with Unpredictable Obstacles

Zhang

et al. 2023

Drones

View full text Add to dashboard Cite

Unmanned Aerial Vehicles (UAVs) can be an important component in the Internet of Things (IoT) ecosystem due to their ability to collect and transmit data from remote and hard-to-reach areas. Ensuring collision-free navigation for these UAVs is crucial in achieving this goal. However, existing UAV collision-avoidance methods face two challenges: conventional path-planning methods are energy-intensive and computationally demanding, while deep reinforcement learning (DRL)-based motion-planning methods are prone to make UAVs trapped in complex environments—especially for long trajectories with unpredictable obstacles—due to UAVs’ limited sensing ability. To address these challenges, we propose a hybrid collision-avoidance method for the real-time navigation of UAVs in complex environments with unpredictable obstacles. We firstly develop a Human-in-the-Loop DRL (HL-DRL) training module for mapless obstacle avoidance and secondly establish a global-planning module that generates a few points as waypoint guidance. Moreover, a novel goal-updating algorithm is proposed to integrate the HL-DRL training module with the global-planning module by adaptively determining the to-be-reached waypoint. The proposed method is evaluated in different simulated environments. Results demonstrate that our approach can rapidly adapt to changes in environments with short replanning time and prevent the UAV from getting stuck in maze-like environments.

show abstract

“…Although neural-network-based methods may be less explainable, they are still preferred, since they do not require rigorous mathematical proofs or tedious theoretical analyses. In the study of Xue et al [21], seven ranging sensors were used for the perception of the environment, and a reinforcement learning approach based on an actor-critic framework was used to the achieve autonomous navigation of the UAV in an unknown environment. Similarly, in Zhang et al's study [22], more than seven laser ranging sensors were used to sense the environment, and an improved TD3-based algorithm was used to realize an autonomous navigation task for a UAV in a multi-obstacle environment.…”

Section: Introductionmentioning

confidence: 99%

Efficient Focus Autoencoders for Fast Autonomous Flight in Intricate Wild Scenarios

Hu,

Li,

Zhuang

et al. 2023

Drones

View full text Add to dashboard Cite

The autonomous navigation of aerial robots in unknown and complex outdoor environments is a challenging problem that typically requires planners to generate collision-free trajectories based on human expert rules for fast navigation. Presently, aerial robots suffer from high latency in acquiring environmental information, which limits the control strategies that the vehicle can implement. In this study, we proposed the SAC_FAE algorithm for high-speed navigation in complex environments using deep reinforcement learning (DRL) policies. Our approach consisted of a soft actor–critic (SAC) algorithm and a focus autoencoder (FAE). Our end-to-end DRL navigation policy enabled a flying robot to efficiently accomplish navigation tasks without prior map information by relying solely on the front-end depth frames and its own pose information. The proposed algorithm outperformed existing trajectory-based optimization approaches at flight speeds exceeding 3 m/s in multiple testing environments, which demonstrates its effectiveness and efficiency.

show abstract

A UAV Navigation Approach Based on Deep Reinforcement Learning in Large Cluttered 3D Environments

Cited by 16 publications

References 39 publications

Semantic-Driven Autonomous Visual Navigation for Unmanned Aerial Vehicles

Semantic-Driven Autonomous Visual Navigation for Unmanned Aerial Vehicles

A Hybrid Human-in-the-Loop Deep Reinforcement Learning Method for UAV Motion Planning for Long Trajectories with Unpredictable Obstacles

Efficient Focus Autoencoders for Fast Autonomous Flight in Intricate Wild Scenarios

Contact Info

Product

Resources

About