Autonomous navigation of UAV by using real-time model-based reinforcement learning

Imanberdiyev, Nursultan; Fu, Changhong; Kayacan, Erdal; Chen, I‐Ming

doi:10.1109/icarcv.2016.7838739

Cited by 111 publications

(51 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The UAV motion model is the basis for completing navigation and target tracking missions. The UAV can be thought as a rigid body with forces and torques applied from the four rotors and gravity [22,23]. In navigation and target tracking scenario, we assume that UAV is flying at a fixed altitude.…”

Section: Uav Motion Modelmentioning

confidence: 99%

UAV Maneuvering Target Tracking in Uncertain Environments Based on Deep Reinforcement Learning and Meta-Learning

Gan

Chen

et al. 2020

Remote Sensing

View full text Add to dashboard Cite

This paper combines deep reinforcement learning (DRL) with meta-learning and proposes a novel approach, named meta twin delayed deep deterministic policy gradient (Meta-TD3), to realize the control of unmanned aerial vehicle (UAV), allowing a UAV to quickly track a target in an environment where the motion of a target is uncertain. This approach can be applied to a variety of scenarios, such as wildlife protection, emergency aid, and remote sensing. We consider a multi-task experience replay buffer to provide data for the multi-task learning of the DRL algorithm, and we combine meta-learning to develop a multi-task reinforcement learning update method to ensure the generalization capability of reinforcement learning. Compared with the state-of-the-art algorithms, namely the deep deterministic policy gradient (DDPG) and twin delayed deep deterministic policy gradient (TD3), experimental results show that the Meta-TD3 algorithm has achieved a great improvement in terms of both convergence value and convergence rate. In a UAV target tracking problem, Meta-TD3 only requires a few steps to train to enable a UAV to adapt quickly to a new target movement mode more and maintain a better tracking effectiveness.

show abstract

Section: Uav Motion Modelmentioning

confidence: 99%

UAV Maneuvering Target Tracking in Uncertain Environments Based on Deep Reinforcement Learning and Meta-Learning

Gan

Chen

et al. 2020

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Imanberdiyev et al [32] developed a model-based RL algorithm called TEXPLORE to efficiently plan trajectories in unknown environments subject to constraints such as battery life. In [33], the authors use a model predictive controller (MPC) to generate training data for an RL controller, thereby guiding the policy search and avoiding the potentially catastrophic early phase before an effective policy is found.…”

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy optimization

Bøhn

Coates

Moe

et al. 2019

2019 International Conference on Unmanned Aircraft Systems (ICUAS)

144

View full text Add to dashboard Cite

Contemporary autopilot systems for unmanned aerial vehicles (UAVs) are far more limited in their flight envelope as compared to experienced human pilots, thereby restricting the conditions UAVs can operate in and the types of missions they can accomplish autonomously. This paper proposes a deep reinforcement learning (DRL) controller to handle the nonlinear attitude control problem, enabling extended flight envelopes for fixed-wing UAVs. A proof-of-concept controller using the proximal policy optimization (PPO) algorithm is developed, and is shown to be capable of stabilizing a fixed-wing UAV from a large set of initial conditions to reference roll, pitch and airspeed values. The training process is outlined and key factors for its progression rate are considered, with the most important factor found to be limiting the number of variables in the observation vector, and including values for several previous time steps for these variables. The trained reinforcement learning (RL) controller is compared to a proportional-integralderivative (PID) controller, and is found to converge in more cases than the PID controller, with comparable performance. Furthermore, the RL controller is shown to generalize well to unseen disturbances in the form of wind and turbulence, even in severe disturbance conditions.

show abstract

“…However, as RL usually requires a lot of trial and error for the learning, having some restrictions on the environment inevitably slows down the training speed. Such characteristics of RL approaches lead to making use of a virtual environment, where the agent can safely make trials and errors quickly without concerning the actual hardware [28,29]. By using sensory data from the accelerometer, it is shown that the RL model can plan swing-free trajectories while carrying suspended load [30].…”

Section: Drone Navigation With Reinforcement Learningmentioning

confidence: 99%

Obstacle Avoidance Drone by Deep Reinforcement Learning and Its Racing with Human Pilot

2019

View full text Add to dashboard Cite

Drones with obstacle avoidance capabilities have attracted much attention from researchers recently. They typically adopt either supervised learning or reinforcement learning (RL) for training their networks. The drawback of supervised learning is that labeling of the massive dataset is laborious and time-consuming, whereas RL aims to overcome such a problem by letting an agent learn with the data from its environment. The present study aims to utilize diverse RL within two categories: (1) discrete action space and (2) continuous action space. The former has the advantage in optimization for vision datasets, but such actions can lead to unnatural behavior. For the latter, we propose a U-net based segmentation model with an actor-critic network. Performance is compared between these RL algorithms with three different environments such as the woodland, block world, and the arena world, as well as racing with human pilots. Results suggest that our best continuous algorithm easily outperformed the discrete ones and yet was similar to an expert pilot.

show abstract

Autonomous navigation of UAV by using real-time model-based reinforcement learning

Cited by 111 publications

References 11 publications

UAV Maneuvering Target Tracking in Uncertain Environments Based on Deep Reinforcement Learning and Meta-Learning

UAV Maneuvering Target Tracking in Uncertain Environments Based on Deep Reinforcement Learning and Meta-Learning

Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy optimization

Obstacle Avoidance Drone by Deep Reinforcement Learning and Its Racing with Human Pilot

Contact Info

Product

Resources

About