A Simulator and First Reinforcement Learning Results for Underwater Mapping

Abstract

Underwater mapping with mobile robots has a wide range of applications, and good models are lacking for key parts of the problem, such as sensor behavior. The specific focus here is the huge environmental problem of underwater litter, in the context of the Horizon 2020 SeaClear project, where a team of robots is being developed to map and collect such litter. No reinforcement-learning solution to underwater mapping has been proposed thus far, even though the framework is well suited for robot control in unknown settings. As a key contribution, this paper therefore makes a first attempt to apply deep reinforcement learning (DRL) to this problem by exploiting two state-of-the-art algorithms and making a number of mapping-specific improvements. Since DRL often requires millions of samples to work, a fast simulator is required, and another key contribution is to develop such a simulator from scratch for mapping seafloor objects with an underwater vehicle possessing a sonar-like sensor. Extensive numerical experiments on a range of algorithm variants show that the best DRL method collects litter significantly faster than a baseline lawn mower trajectory.

A High–Efficiency Side–Scan Sonar Simulator for High–Speed Seabed Mapping

“…A sonar simulator is a numerical simulation technology that can provide a virtual experimental environment to support design, performance prediction, and fault diagnosis, which is especially useful for a new–principle sonar [ 3 , 4 , 5 ]. For instance, Bouxsein modeled the acoustic scattering characteristics of complex geometric surfaces, and then established an underwater simulation environment to explore the underwater automatic obstacle avoidance technology [ 6 ].…”