Towards Using Reinforcement Learning for Autonomous Docking of Unmanned Surface Vehicles

Abstract

Providing full autonomy of Unmanned Surface Vehicles (USV) is a difficult goal to achieve. Autonomous docking is a subtask which is particularly difficult. The vessel has to distinguish between obstacles and the dock, and the obstacles can be either static or moving. In this paper, we developed a simulator using Reinforcement Learning (RL) to approach the problem. We studied several scenarios for the task of docking of a USV in a simulator environment. The scenarios were defined with different sensor inputs and start-stop procedures, but with a simple shared reward function. The results show that the system managed to solve the task when the IMU (Inertial Measurement Unit) and GNSS ( Global Navigation Satellite System) sensors were used to estimate the state, despite the simplicity of the reward function.

Automated Docking for Marine Surface Vessels—A Survey

“…The earliest uses of ANNs were so-called shallow networks, consisting of only one hidden layer, while modern methods use deeper networks with vastly more parameters, at the expense of requiring huge computational resources to train. ANNs were used in 52 of the publications [10], [14], [15], [17], [18], [20], [23], [24], [31]- [35], [37], [38], [43], [44], [49]- [51], [53], [67], [72]- [75], [80], [81], [85], [86], [103], [108], [115], [123], [127], [129], [145], [148], [150], [171], [172], [176], [182], [183], [185], [199], [200], [202], [211], [215], [216], [226].…”