Plankton classification with high-throughput submersible holographic microscopy and transfer learning

Abstract

Background Plankton are foundational to marine food webs and an important feature for characterizing ocean health. Recent developments in quantitative imaging devices provide in-flow high-throughput sampling from bulk volumes—opening new ecological challenges exploring microbial eukaryotic variation and diversity, alongside technical hurdles to automate classification from large datasets. However, a limited number of deployable imaging instruments have been coupled with the most prominent classification algorithms—effectively limiting the extraction of curated observations from field deployments. Holography offers relatively simple coherent microscopy designs with non-intrusive 3-D image information, and rapid frame rates that support data-driven plankton imaging tasks. Classification benchmarks across different domains have been set with transfer learning approaches, focused on repurposing pre-trained, state-of-the-art deep learning models as classifiers to learn new image features without protracted model training times. Combining the data production of holography, digital image processing, and computer vision could improve in-situ monitoring of plankton communities and contribute to sampling the diversity of microbial eukaryotes. Results Here we use a light and portable digital in-line holographic microscope (The HoloSea) with maximum optical resolution of 1.5 μm, intensity-based object detection through a volume, and four different pre-trained convolutional neural networks to classify > 3800 micro-mesoplankton (> 20 μm) images across 19 classes. The maximum classifier performance was quickly achieved for each convolutional neural network during training and reached F1-scores > 89%. Taking classification further, we show that off-the-shelf classifiers perform strongly across every decision threshold for ranking a majority of the plankton classes. Conclusion These results show compelling baselines for classifying holographic plankton images, both rare and plentiful, including several dinoflagellate and diatom groups. These results also support a broader potential for deployable holographic microscopes to sample diverse microbial eukaryotic communities, and its use for high-throughput plankton monitoring.

Combining multi-marker metabarcoding and digital holography to describe eukaryotic plankton across the Newfoundland Shelf

“…These include protists belonging to the major phyla Cercozoa in 18S ASVs, and haptophytes in the 16S ASVs from widely abundant coccolithophorid genera Emiliania and non-coccolithophorid genera Phaeocystis , which forms the densest colonial blooms in the North Atlantic and Southern Oceans 59 . Each group was likely unidentifiable in our holographic images due to their small cell size (< 50 µm) which has previously been difficult for recovering sharp images using the HoloSea 60 , and due to morphologically indistinct features. Metabarcoding also provided potential and validated taxonomic identities of unidentified image objects.…”

“…Quantitative imaging by digital in-line holography captured abundant micro and mesoplankton from every trophic level using the pipeline for hologram reconstruction and object detection outlined in 60 . The HoloSea returned highly focused images for quantitative observations of major plankton clades in aquatic ecology (diatoms, dinoflagellates, ciliates, copepods, radiolarians, silicoflagellates, other large protists) but other important groups (coccolithophorids, haptophytes, other small flagellates, fish larvae) are currently omitted due to size, pixel resolution, or morphologically indistinct features.…”

“…The full dataset includes 27 water samples from 15 stations (See Table S1 ) at 5 m, seven from the SE Grand Banks and eight from Bonavista Banks; three shelf break stations along each transect were also analyzed at 20 and 50 m. First, seawater was pumped at 150 mL min −1 using a peristaltic pump (Fisherbrand GP1000) into a digital in-line holographic microscope, the HoloSea S5 (Described below) at 10 frames s −1 and the seawater was collected in a container for further filtration for DNA extraction. The lower size limit of the object detection algorithm for the HoloSea S5 was set at 20 μm as a conventional lower size threshold to capture microplankton and larger objects 60 . Directly after imaging, samples were pumped through sterile tubing (Masterflex) and filtered onto 10 μm polycarbonate Isopore Membrane filters (Millipore, United States).…”

“…The HoloSea (92 × 351 mm, 2.6 kg) is a digital in-line holographic microscope using a solid-state laser (386 nm) as a point source emitted through a 0.5 µm pinhole located 54 mm distance from the monochrome complementary metal–oxide–semiconductor (CMOS) camera with 7.4 µm per pixel in the resultant 2048 × 2048 hologram. Further specifications and theoretical background for the HoloSea and 4-Deep software are described in 60 . To briefly detail the acquisition process of plankton objects, first raw holograms are reconstructed in 4-Deep Octopus software using a Helmholtz-Kirchhoff transformation and subsequently, using the 4-Deep Stingray software, biological Regions of Interest (ROIs) are detected using a global adaptive thresholding and filtered for in-focus objects in each reconstructed plane 60 .…”

“…Further specifications and theoretical background for the HoloSea and 4-Deep software are described in 60 . To briefly detail the acquisition process of plankton objects, first raw holograms are reconstructed in 4-Deep Octopus software using a Helmholtz-Kirchhoff transformation and subsequently, using the 4-Deep Stingray software, biological Regions of Interest (ROIs) are detected using a global adaptive thresholding and filtered for in-focus objects in each reconstructed plane 60 . These in-focus objects compose our quantitative profiles which were manually identified to the lowest possible taxonomic rank ( 85 Table S2 ).…”