Autonomous Vehicular Landings on the Deck of an Unmanned Surface Vehicle using Deep Reinforcement Learning

Abstract

SummaryAutonomous landing on the deck of a boat or an unmanned surface vehicle (USV) is the minimum requirement for increasing the autonomy of water monitoring missions. This paper introduces an end-to-end control technique based on deep reinforcement learning for landing an unmanned aerial vehicle on a visual marker located on the deck of a USV. The solution proposed consists of a hierarchy of Deep Q-Networks (DQNs) used as high-level navigation policies that address the two phases of the flight: the marker detection and the descending manoeuvre. Few technical improvements have been proposed to stabilize the learning process, such as the combination of vanilla and double DQNs, and a partitioned buffer replay. Simulated studies proved the robustness of the proposed algorithm against different perturbations acting on the marine vessel. The performances obtained are comparable with a state-of-the-art method based on template matching.

Implementation of English “Online and Offline” Hybrid Teaching Recommendation Platform Based on Reinforcement Learning

“…Polvara et al [16] research results show that the idea of reinforcement learning is to maximize the sum of long-term rewards as the goal, through the independent exploration of the unknown environment, in a certain state and action under the influence of the environment and get the corresponding reward and then get more reward adjustment strategy, ultimately in the process of continuous interaction with the environment learning to the optimal strategy.…”

Twin Delayed Deep Deterministic Policy Gradient-Based Target Tracking for Unmanned Aerial Vehicle With Achievement Rewarding and Multistage Training

“…Other approaches have adopted deep reinforcement learning to handle the continuous nature of control. In the work of [33], tracking was used with landing based on decomposition into two separate tasks, namely, marker alignment and vertical descent.…”

Hierarchical Active Tracking Control for UAVs via Deep Reinforcement Learning

“…Manan et al applied RL to teach UAV vision-based guidance tasks [17]. Riccardo et al achieved autonomous landing on the deck of an unmanned surface vehicle, using RL [18]. Through trial and error, model-free deep RL can learn complicated high-level control policies for UAVs, which maps visual observations to action commands directly.…”