Towards Benthic Habitat 3D Mapping Using Machine Learning Algorithms and Structures from Motion Photogrammetry

Abstract

The accurate classification and 3D mapping of benthic habitats in coastal ecosystems are vital for developing management strategies for these valuable shallow water environments. However, both automatic and semiautomatic approaches for deriving ecologically significant information from a towed video camera system are quite limited. In the current study, we demonstrate a semiautomated framework for high-resolution benthic habitat classification and 3D mapping using Structure from Motion and Multi View Stereo (SfM-MVS) algorithms and automated machine learning classifiers. The semiautomatic classification of benthic habitats was performed using several attributes extracted automatically from labeled examples by a human annotator using raw towed video camera image data. The Bagging of Features (BOF), Hue Saturation Value (HSV), and Gray Level Co-occurrence Matrix (GLCM) methods were used to extract these attributes from 3000 images. Three machine learning classifiers (k-nearest neighbor (k-NN), support vector machine (SVM), and bagging (BAG)) were trained by using these attributes, and their outputs were assembled by the fuzzy majority voting (FMV) algorithm. The correctly classified benthic habitat images were then geo-referenced using a differential global positioning system (DGPS). Finally, SfM-MVS techniques used the resulting classified geo-referenced images to produce high spatial resolution digital terrain models and orthophoto mosaics for each category. The framework was tested for the identification and 3D mapping of seven habitats in a portion of the Shiraho area in Japan. These seven habitats were corals (Acropora and Porites), blue corals (H. coerulea), brown algae, blue algae, soft sand, hard sediments (pebble, cobble, and boulders), and seagrass. Using the FMV algorithm, we achieved an overall accuracy of 93.5% in the semiautomatic classification of the seven habitats.

Which Method for Which Purpose? A Comparison of Line Intercept Transect and Underwater Photogrammetry Methods for Coral Reef Surveys

“…Photogrammetry enables the quantitative monitoring of physical (e.g., structural complexity: slope, fractal dimension, surface complexity) and biological features (e.g., cover of benthic communities, colonies size and abundance) of ecosystems over time (e.g., Storlazzi et al, 2016;Fukunaga et al, 2019;Price et al, 2019;Carlot et al, 2020). These new techniques and methods are likely to become new standards for reef surveying in the coming years (Obura et al, 2019;D'Urban et al, 2020) notably with new solutions helping to automate image analysis such as the widely used machine-learning CoralNet tool, which estimates of coral cover are highly comparable to those generated by reef experts (Williams et al, 2019) and other artificial intelligence applications (e.g., González-Rivero et al, 2016;Hopkinson et al, 2020;Mohamed et al, 2020).…”

A protocol for the large‐scale analysis of reefs using Structure from Motion photogrammetry

“…learning/neural network classification (Chirayath & Instrella, 2019;Hopkinson et al, 2020;Mohamed et al, 2020). Additional model outputs (DEM/orthophoto/textured mesh/shapes)…”

“…The technique is also being usefully applied to inform analysis of other marine and coastal systems, using drones and remotely operated vehicles (Casella et al., 2017; Castellanos‐Galindo, Casella, Mejía‐Rentería, & Rovere, 2019; Chirayath & Instrella, 2019; Palma et al., 2018; Price et al., 2019; Teague & Scott, 2017; Varela et al., 2019), making this a rapidly evolving and adaptable tool. The recent application of machine learning and convolutional neural networks to aid habitat/species classification of 3D mapped outputs will likely further widen the scope of this tool (Chirayath & Instrella, 2019; Hopkinson et al., 2020; Mohamed, Nadaoka, & Nakamura, 2020).…”

“…There are a diverse array of outputs and approaches possible for reef ecology using SfM, summarized in Figure 3. A number of methods are now available to analyse the resulting SfM‐derived 3D surface morphometrics, which range in complexity, software cost and user‐training: ArcGIS (Burns et al., 2015; Fukunaga et al., 2019); SLAM/Python‐based (Ferrari, McKinnon, et al, 2016; Friedman, Pizarro, Williams, & Johnson‐Roberson, 2012; González‐Rivero et al., 2017); Fledermaus (Storlazzi, Dartnell, Hatcher, & Gibbs, 2016); Rhino (Young, Dey, Rogers, & Exton, 2017); GeoMagic (Ferrari et al., 2017); Meshlab/Blender (House et al., 2018); R (Schlager, 2019) or machine learning/neural network classification (Chirayath & Instrella, 2019; Hopkinson et al., 2020; Mohamed et al., 2020).…”

3D Fine-scale Terrain Variables from Underwater Photogrammetry: A New Approach to Benthic Microhabitat Modeling in a Circalittoral Rocky Shelf

“…Different algorithms were proposed: a custom multiscale convolutional network [43], texture features extracting and self-organizing maps [44], support vector machines [45] or, more recently, semantic segmentation models [46]. Recent works also benefit from 3D modeling techniques of ocean floors to improve performance, using for example a ResNet152 classifier [47] or even ensembles of several concurrent algorithms [48].…”