ROSEBUD: A Deep Fluvial Segmentation Dataset for Monocular Vision-Based River Navigation and Obstacle Avoidance

Abstract

Obstacle detection for autonomous navigation through semantic image segmentation using neural networks has grown in popularity for use in unmanned ground and surface vehicles because of its ability to rapidly create a highly accurate pixel-wise classification of complex scenes. Due to the lack of available training data, semantic networks are rarely applied to navigation in complex water scenes such as rivers, creeks, canals, and harbors. This work seeks to address the issue by making a one-of-its-kind River Obstacle Segmentation En-Route By USV Dataset (ROSEBUD) publicly available for use in robotic SLAM applications that map water and non-water entities in fluvial images from the water level. ROSEBUD provides a challenging baseline for surface navigation in complex environments using complex fluvial scenes. The dataset contains 549 images encompassing various water qualities, seasons, and obstacle types that were taken on narrow inland rivers and then hand annotated for use in semantic network training. The difference between the ROSEBUD dataset and existing marine datasets was verified. Two state-of-the-art networks were trained on existing water segmentation datasets and tested for generalization to the ROSEBUD dataset. Results from further training show that modern semantic networks custom made for water recognition, and trained on marine images, can properly segment large areas, but they struggle to properly segment small obstacles in fluvial scenes without further training on the ROSEBUD dataset.

Developing an expansion‐based obstacle detection using panoptic segmentation

“…Various studies have employed monocular methods for obstacle detection; initially, conventional image processing techniques were used for obstacle detection in these studies (Badrloo et al, 2022b; Liu, Li, Liu, et al, 2021; Padhy et al, 2019). Recently, studies applied artificial neural networks to real‐time obstacle identification due to insufficient conventional image processing techniques for real‐time applications (Hatch et al, 2021; Lambert et al, 2022; Shi et al, 2023). Among these four monocular techniques, appearance‐based, depth‐based, and expansion‐based methods have been the focus of most of the research that has utilized artificial neural networks for obstacle detection.…”

Aerial Fluvial Image Dataset for Deep Semantic Segmentation Neural Networks and Its Benchmarks

“…Furthermore the distribution of water labeled pixels in satellite imagery is significantly lower than any imagery taken by drones in stable flight at altitude. Maritime [6], [39], [40], [41], [42], river [43], [44], and lakes and canal [45] datasets for ASV navigation and mapping have images collected only from the surface level, which share a common spatial object distribution with all boat or shore-based imagery: water in the lower, potential obstacles in the middle, and sky in the upper portions of images. This spatial pattern of surface images is diametrically opposed to aerial images of fluvial scenes [as shown in Fig.…”