Temporal characteristics of plankton indicators in coastal waters: High-frequency data from PlanktonScope

Bi, Hongsheng; Song, Junting; Zhao, Jian; Liu, Hui; Cheng, Xuemin; Wang, Linlin; Cai, Zhonghua; Benfield, Mark C.; Otto, Saskia A.; Goberville, Éric; Keister, Julie E.; Yang, Yong; Yu, Xin; Cai, Jianfei; Ying, Kezhen; Conversi, Alessandra

doi:10.1016/j.seares.2022.102283

Cited by 11 publications

(7 citation statements)

References 117 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A notable example of our success in this area is the deployment of a PlanktonScope system along the Yangjiang coast since 2021, positioned in front of a nuclear power plant cooling water intake. This system has been instrumental in providing real‐time data on plankton density, with a specific focus on mysid shrimp, which is crucial as mysid swarms could potentially clog cooling water intake (Bi et al 2022). Furthermore, we have also deployed another PlanktonScope system in Chesapeake Bay since February 2023.…”

Section: Discussionmentioning

confidence: 99%

Taming the data deluge: A novel end‐to‐end deep learning system for classifying marine biological and environmental images

Bi,

Cheng,

Cheng

et al. 2023

Limnology & Ocean Methods

Self Cite

View full text Add to dashboard Cite

Underwater imaging enables nondestructive plankton sampling at frequencies, durations, and resolutions unattainable by traditional methods. These systems necessitate automated processes to identify organisms efficiently. Early underwater image processing used a standard approach: binarizing images to segment targets, then integrating deep learning models for classification. While intuitive, this infrastructure has limitations in handling high concentrations of biotic and abiotic particles, rapid changes in dominant taxa, and highly variable target sizes. To address these challenges, we introduce a new framework that starts with a scene classifier to capture large within‐image variation, such as disparities in the layout of particles and dominant taxa. After scene classification, scene‐specific Mask regional convolutional neural network (Mask R‐CNN) models are trained to separate target objects into different groups. The procedure allows information to be extracted from different image types, while minimizing potential bias for commonly occurring features. Using in situ coastal plankton images, we compared the scene‐specific models to the Mask R‐CNN model encompassing all scene categories as a single full model. Results showed that the scene‐specific approach outperformed the full model by achieving a 20% accuracy improvement in complex noisy images. The full model yielded counts that were up to 78% lower than those enumerated by the scene‐specific model for some small‐sized plankton groups. We further tested the framework on images from a benthic video camera and an imaging sonar system with good results. The integration of scene classification, which groups similar images together, can improve the accuracy of detection and classification for complex marine biological images.

show abstract

Section: Discussionmentioning

confidence: 99%

Taming the data deluge: A novel end‐to‐end deep learning system for classifying marine biological and environmental images

Bi,

Cheng,

Cheng

et al. 2023

Limnology & Ocean Methods

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this study, plankton images were captured using PlanktonScope 1 , a shadowgraph imaging system, from the coastal waters of Guangdong, P.R. China (Song et al, 2020;Liu et al, 2021;Bi et al, 2022). PlanktonScope effectively preserves the original shape and size of specimens without distortion (Bi et al, 2012;Song et al, 2020).…”

Section: Data Collection and Descriptionmentioning

confidence: 99%

Automated zooplankton size measurement using deep learning: Overcoming the limitations of traditional methods

Zhang,

Bi,

Wang

et al. 2024

Front. Mar. Sci.

Self Cite

View full text Add to dashboard Cite

Zooplankton size is a crucial indicator in marine ecosystems, reflecting demographic structure, species diversity and trophic status. Traditional methods for measuring zooplankton size, which involve direct sampling and microscopic analysis, are laborious and time-consuming. In situ imaging systems are useful sampling tools; however, the variation in angles, orientations, and image qualities presented considerable challenges to early machine learning models tasked with measuring sizes.. Our study introduces a novel, efficient, and precise deep learning-based method for zooplankton size measurement. This method employs a deep residual network with an adaptation: replacing the fully connected layer with a convolutional layer. This modification allows for the generation of an accurate predictive heat map for size determination. We validated this automated approach against manual sizing using ImageJ, employing in-situ images from the PlanktonScope. The focus was on three zooplankton groups: copepods, appendicularians, and shrimps. An analysis was conducted on 200 individuals from each of the three groups. Our automated method's performance was closely aligned with the manual process, demonstrating a minimal average discrepancy of just 1.84%. This significant advancement presents a rapid and reliable tool for zooplankton size measurement. By enhancing the capacity for immediate and informed ecosystem-based management decisions, our deep learning-based method addresses previous challenges and opens new avenues for research and monitoring in zooplankton.

show abstract

“…The fact that plankton imagery is usually analyzed after the cruise due to the large quantity of data, precludes it from being used for adaptive sampling, which by definition needs near-immediate data availability. With advancements in ocean technology, thanks to the increased affordability and availability of advanced hard-, and software, the number of studies working on real-time identification and adaptive sampling based on different underwater vehicles has increased though in recent years (Fossum et al, 2019;Ohman et al, 2019;Stankiewicz et al, 2021;Bi et al, 2022). However, having the necessary computing power at sea to classify large quantities of videography remains a bottleneck.…”

Section: Introductionmentioning

confidence: 99%

Edge computing at sea: high-throughput classification of in-situ plankton imagery for adaptive sampling

Schmid,

Daprano,

Damle

et al. 2023

Front. Mar. Sci.

View full text Add to dashboard Cite

The small sizes of most marine plankton necessitate that plankton sampling occur on fine spatial scales, yet our questions often span large spatial areas. Underwater imaging can provide a solution to this sampling conundrum but collects large quantities of data that require an automated approach to image analysis. Machine learning for plankton classification, and high-performance computing (HPC) infrastructure, are critical to rapid image processing; however, these assets, especially HPC infrastructure, are only available post-cruise leading to an ‘after-the-fact’ view of plankton community structure. To be responsive to the often-ephemeral nature of oceanographic features and species assemblages in highly dynamic current systems, real-time data are key for adaptive oceanographic sampling. Here we used the new In-situ Ichthyoplankton Imaging System-3 (ISIIS-3) in the Northern California Current (NCC) in conjunction with an edge server to classify imaged plankton in real-time into 170 classes. This capability together with data visualization in a heavy.ai dashboard makes adaptive real-time decision-making and sampling at sea possible. Dual ISIIS-Deep-focus Particle Imager (DPI) cameras sample 180 L s-1, leading to >10 GB of video per min. Imaged organisms are in the size range of 250 µm to 15 cm and include abundant crustaceans, fragile taxa (e.g., hydromedusae, salps), faster swimmers (e.g., krill), and rarer taxa (e.g., larval fishes). A deep learning pipeline deployed on the edge server used multithreaded CPU-based segmentation and GPU-based classification to process the imagery. AVI videos contain 50 sec of data and can contain between 23,000 - 225,000 particle and plankton segments. Processing one AVI through segmentation and classification takes on average 3.75 mins, depending on biological productivity. A heavyDB database monitors for newly processed data and is linked to a heavy.ai dashboard for interactive data visualization. We describe several examples where imaging, AI, and data visualization enable adaptive sampling that can have a transformative effect on oceanography. We envision AI-enabled adaptive sampling to have a high impact on our ability to resolve biological responses to important oceanographic features in the NCC, such as oxygen minimum zones, or harmful algal bloom thin layers, which affect the health of the ecosystem, fisheries, and local communities.

show abstract

Temporal characteristics of plankton indicators in coastal waters: High-frequency data from PlanktonScope

Cited by 11 publications

References 117 publications

Taming the data deluge: A novel end‐to‐end deep learning system for classifying marine biological and environmental images

Taming the data deluge: A novel end‐to‐end deep learning system for classifying marine biological and environmental images

Automated zooplankton size measurement using deep learning: Overcoming the limitations of traditional methods

Edge computing at sea: high-throughput classification of in-situ plankton imagery for adaptive sampling

Contact Info

Product

Resources

About