Demystifying image-based machine learning: a practical guide to automated analysis of field imagery using modern machine learning tools

Abstract

Image-based machine learning methods are becoming among the most widely-used forms of data analysis across science, technology, engineering, and industry. These methods are powerful because they can rapidly and automatically extract rich contextual and spatial information from images, a process that has historically required a large amount of human labor. A wide range of recent scientific applications have demonstrated the potential of these methods to change how researchers study the ocean. However, despite their promise, machine learning tools are still under-exploited in many domains including species and environmental monitoring, biodiversity surveys, fisheries abundance and size estimation, rare event and species detection, the study of animal behavior, and citizen science. Our objective in this article is to provide an approachable, end-to-end guide to help researchers apply image-based machine learning methods effectively to their own research problems. Using a case study, we describe how to prepare data, train and deploy models, and overcome common issues that can cause models to underperform. Importantly, we discuss how to diagnose problems that can cause poor model performance on new imagery to build robust tools that can vastly accelerate data acquisition in the marine realm. Code to perform analyses is provided at https://github.com/heinsense2/AIO_CaseStudy.

Towards standardizing automated image analysis with artificial intelligence for biodiversity

“…Sharing the source codes that accompanies the work described in publications can benefit repeating the analysis and accelerate the follow-up study. However, it is still not as widespread among researchers working in marine science as it is in the broader machine learning community (Belcher et al, 2023). Sometimes, the source code was not provided simultaneously along with the published works on AIbased tools (Belcher et al, 2023), although the source code may be acquired upon request.…”

“…However, it is still not as widespread among researchers working in marine science as it is in the broader machine learning community (Belcher et al, 2023). Sometimes, the source code was not provided simultaneously along with the published works on AIbased tools (Belcher et al, 2023), although the source code may be acquired upon request. The lack of detailed documentation on the released AI-based tools may result in barriers to international cooperation in biodiversity.…”

“…The machine learning community is adopting standards to enable released model sharing via detailed accompanying documentation, including data sources and links to code repositories that deposit the code necessary to repeat the analyses (Mitchell et al, 2019;Belcher et al, 2023). The universally adopted standards would increase the transparency and benefit the reproducibility, and the framework could be used to document any trained machine learning models (Mitchell et al, 2019).…”

“…Previously, what the AI-tools that can be used for analysis of marine field imagery, such as the kinds of information extracted, has been well reviewed (Belcher et al, 2023). The developed AIbased tools improved the efficiency compared with traditional manual strategies and were used for practical applications, such as automated annotation of marine life imagery (Zhang et al, 2023), and detection of megabenthic fauna in optical images from the Clarion-Clipperton Zone (CCZ) in the Central Pacific, where intensive mining exploration are undergoing (Mbani et al, 2023).…”

“…The analysis results may be used as general baseline data to assess the impacts of post-implementation of an activity. To build and use a machine learning-based pipeline, researchers usually need define an analytical task, construct the training and test datasets of the task, select and train the model, evaluate and improve the model performance, and apply to new data of the task (Belcher et al, 2023). Standardizations in these steps could improve the efficiency, reproducibility, and reliability in building AI-based tools.…”