Towards Automated Annotation of Benthic Survey Images: Variability of Human Experts and Operational Modes of Automation

Abstract

Global climate change and other anthropogenic stressors have heightened the need to rapidly characterize ecological changes in marine benthic communities across large scales. Digital photography enables rapid collection of survey images to meet this need, but the subsequent image annotation is typically a time consuming, manual task. We investigated the feasibility of using automated point-annotation to expedite cover estimation of the 17 dominant benthic categories from survey-images captured at four Pacific coral reefs. Inter- and intra- annotator variability among six human experts was quantified and compared to semi- and fully- automated annotation methods, which are made available at coralnet.ucsd.edu. Our results indicate high expert agreement for identification of coral genera, but lower agreement for algal functional groups, in particular between turf algae and crustose coralline algae. This indicates the need for unequivocal definitions of algal groups, careful training of multiple annotators, and enhanced imaging technology. Semi-automated annotation, where 50% of the annotation decisions were performed automatically, yielded cover estimate errors comparable to those of the human experts. Furthermore, fully-automated annotation yielded rapid, unbiased cover estimates but with increased variance. These results show that automated annotation can increase spatial coverage and decrease time and financial outlay for image-based reef surveys.

Automated Image Analysis of Offshore Infrastructure Marine Biofouling

“…Photographic surveys are now considered the standard within marine ecological field studies, due to the ability to collect substantial datasets efficiently, consistently and safely. Ongoing advancements and improvements in underwater digital photography quality, digital storage and computer vision methods will rapidly accelerate the analysis of underwater photography [19].…”

“…MGAs review ROV survey footage Quantification of marine organisms from underwater photography is more challenging than in situ inspection. This is due to limited image resolution, variable lighting conditions, water turbidity and the inability to interact with the organisms [19]. This is particularly true when considering monitoring near-surface offshore infrastructure, due to swell, sea state and light penetration.…”

“…This is particularly true when considering monitoring near-surface offshore infrastructure, due to swell, sea state and light penetration. Analysis of marine growth on offshore platforms is conducted at greater depths and with limited light conditions [20,21], compared to similar studies on shallower coral reefs [19]. Inferior conditions may have an influence on the quality of the images collected.…”

“…Beijbom et al [19] discussed the ongoing advances in underwater image collection, image analysis and the advancement of underwater robotics (e.g., Autonomous Underwater Vehicles (AUVs) and ROV technologies). As such, Beijbom et al [19,22] developed an automated annotation system (CoralNet, https://coralnet.ucsd.edu/) for coral reef survey images, with a publically available, user-friendly interface.…”

“…As such, Beijbom et al [19,22] developed an automated annotation system (CoralNet, https://coralnet.ucsd.edu/) for coral reef survey images, with a publically available, user-friendly interface. The system assesses the texture and colour of a local image patch around randomly allocated annotation points, assigning the point to a predefined list of species or labels, using recent advances in computer vision science [19,22]. Full details of the model, including system algorithms are available in Beijbom et al [19,22].…”