Jellytoring: Real-Time Jellyfish Monitoring Based on Deep Learning Object Detection

Abstract

During the past decades, the composition and distribution of marine species have changed due to multiple anthropogenic pressures. Monitoring these changes in a cost-effective manner is of high relevance to assess the environmental status and evaluate the effectiveness of management measures. In particular, recent studies point to a rise of jellyfish populations on a global scale, negatively affecting diverse marine sectors like commercial fishing or the tourism industry. Past monitoring efforts using underwater video observations tended to be time-consuming and costly due to human-based data processing. In this paper, we present Jellytoring, a system to automatically detect and quantify different species of jellyfish based on a deep object detection neural network, allowing us to automatically record jellyfish presence during long periods of time. Jellytoring demonstrates outstanding performance on the jellyfish detection task, reaching an F1 score of 95.2%; and also on the jellyfish quantification task, as it correctly quantifies the number and class of jellyfish on a real-time processed video sequence up to a 93.8% of its duration. The results of this study are encouraging and provide the means towards a efficient way to monitor jellyfish, which can be used for the development of a jellyfish early-warning system, providing highly valuable information for marine biologists and contributing to the reduction of jellyfish impacts on humans.

Automating Jellyfish Species Recognition through Faster Region-Based Convolution Neural Networks

“…Corresponding results obtained by Martin-Abadal et al [10] similarly rank Cotylorhiza tuberculata (along with Rhizostoma pulmo, which was not considered in the current study) higher than Pelagia noctiluca in terms of correct identification success. These authors attribute this result to the fact that Cotylorhiza tuberculata is a larger jellyfish whose body remains relatively unchanged whilst swimming, thus rendering them more amenable for correct identification, while in Pelagia noctiluca, the relative position of the tentacles in relation to the main body (umbrella) changes to a greater extent with the movement of the animal, adopting a multitude of shapes, making it more difficult to identify.…”

“…This further substantiates the robust performance of the same classification models in correctly identifying jellyfish species from the images provided. The f 1 score metric values obtained in the current study, ranging between 0.843 (Pelagia noctiluca within classification Model 4) to 1 (Velella velella, salps and Cotylorhiza tuberculata within classification Model 5) are comparable to the same metric values reported within [10], which range between 0.936 and 0.952.…”

“…The automated processing of underwater footage, besides static imagery, has also been achieved through the same protocols. For instance, the JellyToring system automatically detects and quantifies different species of jellyfish based on a deep object detection neural network, allowing the user to automatically record jellyfish presence during long periods of time [10]. To date, our University of Malta research group has proposed an image-analysis protocol for the characterisation of Large Microplastics (LMPs) extracted from beach sediment [11].…”

Life beneath the ice: jellyfish and ctenophores from the Ross Sea, Antarctica, with an image-based training set for machine learning

“…The use of machine learning for image and video annotation of gelatinous zooplankton remains, however, scarce and most of these first studies could not differentiate between jellyfish species ( Kim et al 2016 , Rife and Rock 2003 ). The few image-based machine-learning studies that could differentiate between some jellyfish species included the detection of moon jellyfish through underwater sonar imagery ( French et al 2019 ) and a real-time jellyfish monitoring tool for three Mediterranean jellyfish species using a deep learning object detection-based neutral network ( Martin-Abadal et al 2020 ). As the future of studying gelatinous zooplankton through in situ optical methods certainly lies in the development of more efficient and accurate video/image analysis tools, with machine-learning-based algorithms able to distinguish between the numerous species, an additional difficulty is providing an accurate training dataset.…”

Sensing Occupancy through Software: Smart Parking Proof of Concept

“…The ideal classifier will have the unit area under the curve and a worst case classifier will have FPR = 100% and TPR = 0 [13]. • Average Precision-it is the measure that considers both Recall and Precision and can be expressed as a function p(r) of the recall and it is given with [82]:…”