The Effect of Physics-Based Corrections and Data Augmentation on Transfer Learning for Segmentation of Benthic Imagery

Walker, Jenny; Yamada, Takaki; Prügel-Bennett, Adam; Thornton, Blair

doi:10.1109/ut.2019.8734463

Cited by 7 publications

(7 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Looking more closely at Figure 9d shows that some of the results of the search do not include crabs, but instead contain other types of benthic organisms. To obtain a more precise result for these categories, supervised learning based approaches are more appropriate (Walker, Yamada, Prugel‐Bennett, & Thornton, 2019). The proposed content based image search may be useful to reduce the effort required for manual annotation by filtering out candidate images that are more likely to contain the targets of interests.…”

Section: Methodsmentioning

confidence: 99%

Learning features from georeferenced seafloor imagery with location guided autoencoders

Yamada

Prügel-Bennett

Thornton

2020

Journal of Field Robotics

Self Cite

View full text Add to dashboard Cite

Although modern machine learning has the potential to greatly speed up the interpretation of imagery, the varied nature of the seabed and limited availability of expert annotations form barriers to its widespread use in seafloor mapping applications. This motivates research into unsupervised methods that function without large databases of human annotations. This paper develops an unsupervised feature learning method for georeferenced seafloor visual imagery that considers patterns both within the footprint of a single image frame and broader scale spatial characteristics. Features within images are learnt using an autoencoder developed based on the AlexNet deep convolutional neural network. Features larger than each image frame are learnt using a novel loss function that regularises autoencoder training using the Kullback–Leibler divergence function to loosely assume that images captured within a close distance of each other look more similar than those that are far away. The method is used to semantically interpret images taken by an autonomous underwater vehicle at the Southern Hydrates Ridge, an active gas hydrate field and site of a seafloor cabled observatory at a depth of 780 m. The method's performance when applied to clustering and content‐based image retrieval is assessed against a ground truth consisting of more than 18,000 human annotations. The study shows that the location based loss function increases the rate of information retrieval by a factor of two for seafloor mapping applications. The effects of physics‐based colour correction and image rescaling are also investigated, showing that the improved consistency of spatial information achieved by rescaling is beneficial for recognising artificial objects such as cables and infrastructures, but is less effective for natural objects that have greater dimensional variability.

show abstract

Section: Methodsmentioning

confidence: 99%

Learning features from georeferenced seafloor imagery with location guided autoencoders

Yamada

Prügel-Bennett

Thornton

2020

Journal of Field Robotics

Self Cite

View full text Add to dashboard Cite

show abstract

“…Furthermore, our approach uses histogram matching to correct for uneven scene brightness among the photos, which is simple, parallelizable, and does not even require knowledge of parameters required for reconstruction of the path of light rays through the water column e.g. as demanded by physics-based approaches 24,45,46 , or photogrammetric structure from-motion 47 and simultaneous localization and mapping (SLAM) 48,49 . Moreover, performing color normalization on the raw images improved the accuracy of seafloor classification; this is similar to observations by previous works such as by 24,45 .…”

Section: Discussionmentioning

confidence: 99%

“…as demanded by physics-based approaches 24,45,46 , or photogrammetric structure from-motion 47 and simultaneous localization and mapping (SLAM) 48,49 . Moreover, performing color normalization on the raw images improved the accuracy of seafloor classification; this is similar to observations by previous works such as by 24,45 . Particularly in our case, the raw images acquired at varying altitude were not directly comparable; these images represent regions of the seafloor with varying spatial foot print and scene brightness.…”

Section: Discussionmentioning

confidence: 99%

Implementation of an automated workflow for image-based seafloor classification with examples from manganese-nodule covered seabed areas in the Central Pacific Ocean

Mbani

Schoening

Gazis

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Mapping and monitoring of seafloor habitats are key tasks for fully understanding ocean ecosystems and resilience, which contributes towards sustainable use of ocean resources. Habitat mapping relies on seafloor classification typically based on acoustic methods, and ground truthing through direct sampling and optical imaging. With the increasing capabilities to record high-resolution underwater images, manual approaches for analyzing these images to create seafloor classifications are no longer feasible. Automated workflows have been proposed as a solution, in which algorithms assign pre-defined seafloor categories to each image. However, in order to provide consistent and repeatable analysis, these automated workflows need to address e.g., underwater illumination artefacts, variances in resolution and class-imbalances, which could bias the classification. Here, we present a generic implementation of an Automated and Integrated Seafloor Classification Workflow (AI-SCW). The workflow aims to classify the seafloor into habitat categories based on automated analysis of optical underwater images with only minimal amount of human annotations. AI-SCW incorporates laser point detection for scale determination and color normalization. It further includes semi-automatic generation of the training data set for fitting the seafloor classifier. As a case study, we applied the workflow to an example seafloor image dataset from the Belgian and German contract areas for Manganese-nodule exploration in the Pacific Ocean. Based on this, we provide seafloor classifications along the camera deployment tracks, and discuss results in the context of seafloor multibeam bathymetry. Our results show that the seafloor in the Belgian area predominantly comprises densely distributed nodules, which are intermingled with qualitatively larger-sized nodules at local elevations and within depressions. On the other hand, the German area primarily comprises nodules that only partly cover the seabed, and these occur alongside turned-over sediment (artificial seafloor) that were caused by the settling plume following a dredging experiment conducted in the area.

show abstract

“…This process would allow labelling effort for images from one AUV survey to be migrated to a new survey with new operating parameters arising from changes to cameras, lighting or imaging altitude. Classification performance will also be improved by scaling images so that images have the same spatial resolution [13] and adding an efficient bilinear pooling layer to generate more discriminative features [10]. Figure 1 illustrates the proposed framework.…”

Section: Introductionmentioning

confidence: 99%

Improved Benthic Classification using Resolution Scaling and SymmNet Unsupervised Domain Adaptation

Doig¹,

Pizarro²,

Williams³

2023

Preprint

View full text Add to dashboard Cite

Autonomous Underwater Vehicles (AUVs) conduct regular visual surveys of marine environments to characterise and monitor the composition and diversity of the benthos. The use of machine learning classifiers for this task is limited by the low numbers of annotations available and the many fine-grained classes involved. In addition to these challenges, there are domain shifts between image sets acquired during different AUV surveys due to changes in camera systems, imaging altitude, illumination and water column properties leading to a drop in classification performance for images from a different survey where some or all these elements may have changed. This paper proposes a framework to improve the performance of a benthic morphospecies classifier when used to classify images from a different survey compared to the training data. We adapt the SymmNet state-of-the-art Unsupervised Domain Adaptation method with an efficient bilinear pooling layer and image scaling to normalise spatial resolution, and show improved classification accuracy. We test our approach on two datasets with images from AUV surveys with different imaging payloads and locations. The results show that generic domain adaptation can be enhanced to produce a significant increase in accuracy for images from an AUV survey that differs from the training images.

show abstract

The Effect of Physics-Based Corrections and Data Augmentation on Transfer Learning for Segmentation of Benthic Imagery

Cited by 7 publications

References 5 publications

Learning features from georeferenced seafloor imagery with location guided autoencoders

Learning features from georeferenced seafloor imagery with location guided autoencoders

Implementation of an automated workflow for image-based seafloor classification with examples from manganese-nodule covered seabed areas in the Central Pacific Ocean

Improved Benthic Classification using Resolution Scaling and SymmNet Unsupervised Domain Adaptation

Contact Info

Product

Resources

About