Survey on deep learning based computer vision for sonar imagery

Steiniger, Yannik; Kraus, Dieter; Meisen, Tobias

doi:10.1016/j.engappai.2022.105157

Cited by 45 publications

(23 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These models require many image-label pairs to train. Open-source datasets of SSS imagery and substrate labels are not available (Steiniger et al, 2022), therefore this study first created the datasets using open-source software (i.e., PING-Mapper (Bodine and Buscombe, 2022), Doodler , and Make Sense (Skalski, 2019)) from recreation-grade sonar datasets collected on the Pearl and Pascagoula river systems in Mississippi. Segmentation models were then fit to the datasets with Segmentation Gym .…”

Section: Methodsmentioning

confidence: 99%

“…Evaluating the use of semantic segmentation models on sonar imagery has been limited by the fact that there are no open-source datasets available, to our knowledge, necessitating the creation of project-specific datasets (Steiniger et al, 2022). As a result, comparisons between existing methods is difficult as there is no common benchmark.…”

Section: First Open-source Sonar Image-label Datasetsmentioning

confidence: 99%

“…Unsupervised approaches to calculating texture metrics from the imagery have been made available in open-source toolsets (Buscombe et al, 2016;Buscombe, 2017;Hamill et al, 2018), however it is unclear how texture relates to substrate. Supervised approaches, particularly deep learning for semantic segmentation, have recently been used to characterize SSS datasets collected with survey-grade and recreation-grade sonar instruments (Steiniger et al, 2022). Deep learning based approaches have included segmenting the water column (Zheng et al, 2021), prominent lines (Wu et al, 2019;Wang et al, 2020), discriminating objects from sonar shadows and background (Song et al, 2021), sand waves (Yu et al, 2019;Li et al, 2019;Nian et al, 2021), and seagrass and potholes (Rahnemoonfar and Dobbs, 2019).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Open-source approach for reproducible substrate mapping using semantic segmentation on recreation-grade side scan sonar datasets

Bodine,

Buscombe,

Hocking

2023

Preprint

View full text Add to dashboard Cite

Knowledge of the variation and distribution of substrates at large spatial extents in aquatic systems, particularly rivers, is severely lacking, impeding species conservation and ecosystem restoration efforts. Air and space-borne remote sensing important for terrestrial and atmospheric measurements are limited in benthic environments due to river stage, turbidity, and canopy cover, requiring direct observation of conditions in the field. Recreation-grade side scan sonar (SSS) instruments, or fishfinders, have demonstrated their unparalleled value as a low-cost scientific instrument capable of rapidly imaging benthic environments due to the ease of deploying and operating the instrument. However, existing methods for generating georeferenced datasets from these instruments, including sonar mosaics and substrate maps, remains a barrier of adoption for wider scientific inquiry due to the high degree of human-intervention and expertise required to generate these datasets. To address this short-coming, we introduced PING-Mapper; an open-source and freely available Python-based software for generating geospatial benthic datasets from popular Humminbird® instruments reproducibly, with minimal intervention from the user. The previously released Version 1.0 of the software provided automated workflows for exporting georeferenced sonar imagery. This study extends functionality with Version 2.0 by incorporating semantic segmentation with deep neural network models to reproducibly map substrates at large spatial extents. We present a novel approach for generating label-ready sonar datasets, creating label-image training sets, and model training with transfer learning; all with readily available open-source tools. The substrate models, achieving overall accuracies of 78%, are integrated into PING-Mapper v2.0, providing an automated workflow to generate map substrate distribution anywhere. Additional workflows enable masking sonar shadows, calculating independent bedpicks, and correcting attenuation effects in the imagery to improve interpretability. This software provides an improved mechanism for generating geospatial benthic datasets from recreation-grade SSS systems, thereby lowering the barrier for inclusion in wider aquatic research.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: First Open-source Sonar Image-label Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Open-source approach for reproducible substrate mapping using semantic segmentation on recreation-grade side scan sonar datasets

Bodine,

Buscombe,

Hocking

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…At present, the research based on this approach mainly focuses on target detection [8][9][10][11][12][13][14][15]. For instance, [16] proposed a convolutional neural network transfer-learning recognition method using an improved VGG-16 as the framework. The method was able to perform automatic image recognition of side-scan sonar seabed shipwrecks and achieve significantly better accuracy and efficiency than the classical machine-learning SVM algorithm.…”

Section: Introductionmentioning

confidence: 99%

Side-scan Sonar Underwater Target Segmentation Using the BHP-UNet

Tang

Wei

et al. 2023

Preprint

View full text Add to dashboard Cite

Although target detection algorithms based on deep learning have achieved good results in the detection of side-scan sonar underwater targets, their false and missed detection rates are high for multiple densely arranged and overlapping underwater targets. To address this problem, a side-scan sonar underwater target segmentation model based on the Blended Hybrid dilated convolution and Pyramid split attention U-Net (BHP-UNet) algorithm is proposed in this paper. First, the blended hybrid dilated convolution module is adopted to improve the ability of the model to learn deep semantics and shallow features while improving the receptive field. Second, the pyramid split attention module is introduced to establish a long-term dependency between global and local information while processing multi-scale spatial features. Three sets of experimental results show that the BHP-UNet model proposed in this paper has better segmentation performance than the conventional fully convolutional network, U-Net, and DeepLabv3+ models, and it is able to segment dense and overlapping targets to a certain extent. The proposed model will have significance as a guide for practical applications.

show abstract

“…Reinforcement learning learns in an interactive environment by trial and error, using feedback from its own actions and experiences [ 1 ]. A previous paper [ 9 ] considers that deep learning approaches to automatic target recognition should be considered as follows: feature extraction (FE), classification, detection, and segmentation. Since FE is included in classification models, the size and class of the MLO is simultaneously predicted, a situation that includes classification in the detection step.…”

Section: Introductionmentioning

confidence: 99%

Sea Mine Detection Framework Using YOLO, SSD and EfficientDet Deep Learning Models

Munteanu

Diana

Zamfir

et al. 2022

Sensors

View full text Add to dashboard Cite

In the context of new geopolitical tensions due to the current armed conflicts, safety in terms of navigation has been threatened due to the large number of sea mines placed, in particular, within the sea conflict areas. Additionally, since a large number of mines have recently been reported to have drifted into the territories of the Black Sea countries such as Romania, Bulgaria Georgia and Turkey, which have intense commercial and tourism activities in their coastal areas, the safety of those economic activities is threatened by possible accidents that may occur due to the above-mentioned situation. The use of deep learning in a military operation is widespread, especially for combating drones and other killer robots. Therefore, the present research addresses the detection of floating and underwater sea mines using images recorded from cameras (taken from drones, submarines, ships and boats). Due to the low number of sea mine images, the current research used both an augmentation technique and synthetic image generation (by overlapping images with different types of mines over water backgrounds), and two datasets were built (for floating mines and for underwater mines). Three deep learning models, respectively, YOLOv5, SSD and EfficientDet (YOLOv5 and SSD for floating mines and YOLOv5 and EfficientDet for underwater mines), were trained and compared. In the context of using three algorithm models, YOLO, SSD and EfficientDet, the new generated system revealed high accuracy in object recognition, namely the detection of floating and anchored mines. Moreover, tests carried out on portable computing equipment, such as Raspberry Pi, illustrated the possibility of including such an application for real-time scenarios, with the time of 2 s per frame being improved if devices use high-performance cameras.

show abstract

Survey on deep learning based computer vision for sonar imagery

Cited by 45 publications

References 38 publications

Open-source approach for reproducible substrate mapping using semantic segmentation on recreation-grade side scan sonar datasets

Open-source approach for reproducible substrate mapping using semantic segmentation on recreation-grade side scan sonar datasets

Side-scan Sonar Underwater Target Segmentation Using the BHP-UNet

Sea Mine Detection Framework Using YOLO, SSD and EfficientDet Deep Learning Models

Contact Info

Product

Resources

About