A Hybrid Human-in-the-Loop Deep Reinforcement Learning Method for UAV Motion Planning for Long Trajectories with Unpredictable Obstacles

Zhang, Sitong; Li, Yibing; Ye, Fang; Geng, Xiaoyu; Zhou, Zitao; Shi, Tao

doi:10.3390/drones7050311

Cited by 11 publications

(4 citation statements)

References 58 publications

(108 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ref. [36] took advantage of the traditional collision avoidance method and DRL method, resulting in long trajectory planning with unknown obstacles. In [37], UAVs serve in a warehouse for stock inventory, updating real-time paths with image recording.…”

Section: Related Workmentioning

confidence: 99%

Iterative Trajectory Planning and Resource Allocation for UAV-Assisted Emergency Communication with User Dynamics

Zhang,

Wang,

Luo

et al. 2024

Drones

View full text Add to dashboard Cite

The demand for air-to-ground communication has surged in recent years, underscoring the significance of unmanned aerial vehicles (UAVs) in enhancing mobile communication, particularly in emergency scenarios due to their deployment efficiency and flexibility. In situations such as emergency cases, UAVs can function as efficient temporary aerial base stations and enhance communication quality in instances where terrestrial base stations are incapacitated. Trajectory planning and resource allocation of UAVs continue to be vital techniques, while a relatively limited number of algorithms account for the dynamics of ground users. This paper focuses on emergency communication scenarios such as earthquakes, proposing an innovative path planning and resource allocation algorithm. The algorithm leverages a multi-stage subtask iteration approach, inspired by the block coordinate descent technique, to address the challenges presented in such critical environments. In this study, we establish an air-to-ground communication model, subsequently devising a strategy for user dynamics. This is followed by the introduction of a joint scheduling process for path and resource allocation, named ISATR (iterative scheduling algorithm of trajectory and resource). This process encompasses highly interdependent decision variables, such as location, bandwidth, and power resources. For mobile ground users, we employ the cellular automata (CA) method to forecast the evacuation trajectory. This algorithm successfully maintains data communication in the emergency-stricken area and enhances the communication quality through bandwidth division and power control which varies with time. The effectiveness of our algorithm is validated by evaluating the average throughput with different parameters in various simulation conditions and by using several heuristic methods as a contrast.

show abstract

Section: Related Workmentioning

confidence: 99%

Iterative Trajectory Planning and Resource Allocation for UAV-Assisted Emergency Communication with User Dynamics

Zhang,

Wang,

Luo

et al. 2024

Drones

View full text Add to dashboard Cite

show abstract

“…A collision-free, smooth, and dynamically feasible trajectory guarantees flight safety, which is generated by a motion planning module in working scenarios. Motion planning is generally divided into two parts: front-end discrete path search and back-end continuous trajectory optimization, aiming at generating a reference trajectory that satisfies the above three basic conditions for the controller to track [3,4]. Since quadrotor UAVs with small fuselages have limited computational resources of onboard computers, a major research objective is how to fully utilize the limited resources to generate front-end paths and back-end trajectories.…”

Section: Introductionmentioning

confidence: 99%

Position Checking-Based Sampling Approach Combined with Attraction Point Local Optimization for Safe Flight of UAVs

Zhu,

Li,

Tong

et al. 2024

Sensors

View full text Add to dashboard Cite

Trading off the allocation of limited computational resources between front-end path generation and back-end trajectory optimization plays a key role in improving the efficiency of unmanned aerial vehicle (UAV) motion planning. In this paper, a sampling-based kinodynamic planning method that can reduce the computational cost as well as the risks of UAV flight is proposed. Firstly, an initial trajectory connecting the start and end points without considering obstacles is generated. Then, a spherical space is constructed around the topological vertices of the environment, based on the intersections of the trajectory with the obstacles. Next, some unnecessary sampling points, as well as node rewiring, are discarded by the designed position-checking strategy to minimize the computational cost and reduce the risks of UAV flight. Finally, in order to make the planning framework adaptable to complex scenarios, the strategies for selecting different attraction points according to the environment are designed, which further ensures the safe flight of the UAV while improving the success rate of the front-end trajectory. Simulations and real-world experiment comparisons are conducted on a vision-based platform to verify the performance of the proposed method.

show abstract

“…UAVs realize autonomous positioning through the information fusion of an airborne visual sensor and IMU, and then realize autonomous navigation capabilities such as obstacle-avoidance flight partly through the airborne autonomous path planning algorithm. For example, reference [ 18 ] realized obstacle avoidance of a UAV in a dynamic environment based on point-cloud image; reference [ 19 ] adopted deep reinforcement learning to realize an end-to-end obstacle-avoidance decision of UAVs. The advantage of navigation decision planning based on airborne autonomous sensors and processors is that the real-time requirement can be satisfied.…”

Section: Introductionmentioning

confidence: 99%

End-Cloud Collaboration Navigation Planning Method for Unmanned Aerial Vehicles Used in Small Areas

Huagang

et al. 2023

Sensors

View full text Add to dashboard Cite

Unmanned aerial vehicle (UAV) collaboration has become the main means of indoor and outdoor regional search, railway patrol, and other tasks, and navigation planning is one of the key, albeit difficult, technologies. The purpose of UAV navigation planning is to plan reasonable trajectories for UAVs to avoid obstacles and reach the task area. Essentially, it is a complex optimization problem that requires the use of navigation planning algorithms to search for path-point solutions that meet the requirements under the guide of objective functions and constraints. At present, there are autonomous navigation modes of UAVs relying on airborne sensors and navigation control modes of UAVs relying on ground control stations (GCSs). However, due to the limitation of airborne processor computing power, and background command and control communication delay, a navigation planning method that takes into account accuracy and timeliness is needed. First, the navigation planning architecture of UAVs of end-cloud collaboration was designed. Then, the background cloud navigation planning algorithm of UAVs was designed based on the improved particle swarm optimization (PSO). Next, the navigation control algorithm of the UAV terminals was designed based on the multi-objective hybrid swarm intelligent optimization algorithm. Finally, the computer simulation and actual indoor-environment flight test based on small rotor UAVs were designed and conducted. The results showed that the proposed method is correct and feasible, and can improve the effectiveness and efficiency of navigation planning of UAVs.

show abstract

A Hybrid Human-in-the-Loop Deep Reinforcement Learning Method for UAV Motion Planning for Long Trajectories with Unpredictable Obstacles

Cited by 11 publications

References 58 publications

Iterative Trajectory Planning and Resource Allocation for UAV-Assisted Emergency Communication with User Dynamics

Iterative Trajectory Planning and Resource Allocation for UAV-Assisted Emergency Communication with User Dynamics

Position Checking-Based Sampling Approach Combined with Attraction Point Local Optimization for Safe Flight of UAVs

End-Cloud Collaboration Navigation Planning Method for Unmanned Aerial Vehicles Used in Small Areas

Contact Info

Product

Resources

About