Assessing the Image Concept Drift at the OBSEA Coastal Underwater Cabled Observatory

Abstract

The marine science community is engaged in the exploration and monitoring of biodiversity dynamics, with a special interest for understanding the ecosystem functioning and for tracking the growing anthropogenic impacts. The accurate monitoring of marine ecosystems requires the development of innovative and effective technological solutions to allow a remote and continuous collection of data. Cabled fixed observatories, equipped with camera systems and multiparametric sensors, allow for a non-invasive acquisition of valuable datasets, at a high-frequency rate and for periods extended in time. When large collections of visual data are acquired, the implementation of automated intelligent services is mandatory to automatically extract the relevant biological information from the gathered data. Nevertheless, the automated detection and classification of streamed visual data suffer from the “concept drift” phenomenon, consisting of a drop of performance over the time, mainly caused by the dynamic variation of the acquisition conditions. This work quantifies the degradation of the fish detection and classification performance on an image dataset acquired at the OBSEA cabled video-observatory over a one-year period and finally discusses the methodological solutions needed to implement an effective automated classification service operating in real time.

Image dataset for benchmarking automated fish detection and classification algorithms

“…The manually tagged fish individuals for each image make the dataset a valuable benchmark for the multidisciplinary marine science community consisting of biologists, oceanographers, and a growing community of computer scientists and mathematicians skilled in Artificial Intelligence and data science. Methodological comparison could be not only specifically conceived for fish detection and classification, such as Fish4Knowledge 35 , but also for the emerging approaches for active and incremental learning 36 – 38 , or for techniques aimed at mitigating the “Concept Drift” phenomenon, when the classification performance drop for varying species assemblages at changing environmental conditions and training need to be updated 39 – 42 .…”

“…Finally, the reported dataset of labelled images is worthwhile for global image repositories that aim to reduce annotation effort, such as Fathomnet 43 , and, thanks to the tags and the bounding boxes associated to each individual, it can be easily split into training, validation, and test subsets (e.g., K-fold Cross-validation) in order to fit the needs of the specific image analysis algorithm used on the image dataset 32 , 42 , 44 – 47 .…”

Machine learning applied to big data from marine cabled observatories: A case study of sablefish monitoring in the NE Pacific

“…There are available tools to automatically process large quantities of videos for extracting biological data. A number of methodologies have been proposed for fish species recognition and classification over the last two decades, but the great variability of either species morphologies, or the conditions in which the videos are captured, is still a major challenge for automated processing (e.g., Matabos et al, 2017;Marini et al, 2018a;Marini et al, 2018b;Ottaviani et al, 2022). These automated approaches span a wide range of topics within the AI and computer vision based literature (e.g., Hsiao et al, 2014;Nishida et al, 2014;Wong et al, 2015;Chuang et al, 2016;Tills et al, 2018;Harrison et al, 2021;Yang et al, 2021;Liu et al, 2021;Sokolova et al, 2021a;Sokolova et al, 2021b).…”

“…However, those initial attempts highlighted that the fish detection algorithms were still rudimentary, being significantly outperformed by trained human eyes, and therefore calling for significant improvement. Nevertheless, the recent developments based on Deep Learning (DL) Convolutional Neural Network (CNN) processing methods demonstrated high accuracy performance and reliability in completing fish recognition and classification tasks (Konovalov et al, 2019;Lopez-Vazquez et al, 2020;Yang et al, 2021;Lopez-Marcano et al, 2021;Zhao et al, 2021;Ottaviani et al, 2022).…”

“…DL has recently emerged as an innovative field in AI for language processing, computer vision, and the alike (Goodfellow et al, 2016;Han et al, 2020;Langenkämper et al, 2020;Malde et al, 2020;Ottaviani et al, 2022). CNNs are successfully used in computer vision because, thanks to their interconnected structure, they can automatically capture many hidden features of the input data, generating information on a number of valuable image features (shape, texture, and so on) with little manual intervention (Simonyan and Zisserman, 2015;LeCun et al, 2015;Girshick, 2015;Ren et al, 2015).…”