Toward End-to-End Control for UAV Autonomous Landing via Deep Reinforcement Learning

Polvara, Riccardo; Patacchiola, Massimiliano; Sharma, Sanjay; Wan, Jian; Manning, Andrew J.; Sutton, Robert I.; Cangelosi, Angelo

doi:10.1109/icuas.2018.8453449

Cited by 73 publications

(41 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their controller generalizes to avoid multiple obstacles, compared to the singular obstacle avoided by the MPC in training, does not require full state information like the MPC does, and is computed at a fraction of the time. With the advent of DRL, it has also been used for more advanced tasks such as enabling intelligent cooperation between multiple UAVs [34], and for specific control problems such as landing [35]. RL algorithms have also been proposed for attitude control of other autonomous vehicles, including satellites [36] and underwater vehicles.…”

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy optimization

Bøhn

Coates

Moe

et al. 2019

2019 International Conference on Unmanned Aircraft Systems (ICUAS)

144

View full text Add to dashboard Cite

Contemporary autopilot systems for unmanned aerial vehicles (UAVs) are far more limited in their flight envelope as compared to experienced human pilots, thereby restricting the conditions UAVs can operate in and the types of missions they can accomplish autonomously. This paper proposes a deep reinforcement learning (DRL) controller to handle the nonlinear attitude control problem, enabling extended flight envelopes for fixed-wing UAVs. A proof-of-concept controller using the proximal policy optimization (PPO) algorithm is developed, and is shown to be capable of stabilizing a fixed-wing UAV from a large set of initial conditions to reference roll, pitch and airspeed values. The training process is outlined and key factors for its progression rate are considered, with the most important factor found to be limiting the number of variables in the observation vector, and including values for several previous time steps for these variables. The trained reinforcement learning (RL) controller is compared to a proportional-integralderivative (PID) controller, and is found to converge in more cases than the PID controller, with comparable performance. Furthermore, the RL controller is shown to generalize well to unseen disturbances in the form of wind and turbulence, even in severe disturbance conditions.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy optimization

Bøhn

Coates

Moe

et al. 2019

2019 International Conference on Unmanned Aircraft Systems (ICUAS)

144

View full text Add to dashboard Cite

show abstract

“…Learning-based control methods for autonomous landing have also been studied to achieve the optimal control policy under uncertainties. Polvara et al [21] have proposed an approach based on a hierarchy of deep Q-networks (DQNs) that can be used as a high-end control policy for the navigation in different phases. With an optimal policy, they have demonstrated a quadcopter autonomously landing in a large variety of simulated environments.…”

Section: Related Workmentioning

confidence: 99%

Autonomous Landing of a UAV on a Moving Platform Using Model Predictive Control

Feng

Zhang

Baek

et al. 2018

Drones

View full text Add to dashboard Cite

Developing methods for autonomous landing of an unmanned aerial vehicle (UAV) on a mobile platform has been an active area of research over the past decade, as it offers an attractive solution for cases where rapid deployment and recovery of a fleet of UAVs, continuous flight tasks, extended operational ranges, and mobile recharging stations are desired. In this work, we present a new autonomous landing method that can be implemented on micro UAVs that require high-bandwidth feedback control loops for safe landing under various uncertainties and wind disturbances. We present our system architecture, including dynamic modeling of the UAV with a gimbaled camera, implementation of a Kalman filter for optimal localization of the mobile platform, and development of model predictive control (MPC), for guidance of UAVs. We demonstrate autonomous landing with an error of less than 37 cm from the center of a mobile platform traveling at a speed of up to 12 m/s under the condition of noisy measurements and wind disturbances.

show abstract

“…However, filling the gap between real and simulated experiences is everything but simple. To reduce this gap, we built on top of our previous work [12], and we used domain randomization (DR) [10] to improve the generalization capabilities of the DQNs via random sampling of training environments. We show that, when the variability is large enough, the networks learn to generalize well across a large variety of unseen scenarios, including real ones.…”

Section: Introductionmentioning

confidence: 99%

Sim-to-Real Quadrotor Landing via Sequential Deep Q-Networks and Domain Randomization

et al. 2020

Self Cite

View full text Add to dashboard Cite

The autonomous landing of an Unmanned Aerial Vehicle (UAV) on a marker is one of the most challenging problems in robotics. Many solutions have been proposed, with the best results achieved via customized geometric features and external sensors. This paper discusses for the first time the use of deep reinforcement learning as an end-to-end learning paradigm to find a policy for UAVs autonomous landing. Our method is based on a divide-and-conquer paradigm that splits a task into sequential sub-tasks, each one assigned to a Deep Q-Network (DQN), hence the name Sequential Deep Q-Network (SDQN). Each DQN in an SDQN is activated by an internal trigger, and it represents a component of a high-level control policy, which can navigate the UAV towards the marker. Different technical solutions have been implemented, for example combining vanilla and double DQNs, and the introduction of a partitioned buffer replay to address the problem of sample efficiency. One of the main contributions of this work consists in showing how an SDQN trained in a simulator via domain randomization, can effectively generalize to real-world scenarios of increasing complexity. The performance of SDQNs is comparable with a state-of-the-art algorithm and human pilots while being quantitatively better in noisy conditions.

show abstract

Toward End-to-End Control for UAV Autonomous Landing via Deep Reinforcement Learning

Cited by 73 publications

References 24 publications

Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy optimization

Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy optimization

Autonomous Landing of a UAV on a Moving Platform Using Model Predictive Control

Sim-to-Real Quadrotor Landing via Sequential Deep Q-Networks and Domain Randomization

Contact Info

Product

Resources

About