Evaluating Very Deep Convolutional Neural Networks for Nucleus Segmentation from Brightfield Cell Microscopy Images

Abstract

Advances in microscopy have increased output data volumes, and powerful image analysis methods are required to match. In particular, finding and characterizing nuclei from microscopy images, a core cytometry task, remains difficult to automate. While deep learning models have given encouraging results on this problem, the most powerful approaches have not yet been tested for attacking it. Here, we review and evaluate state-of-the-art very deep convolutional neural network architectures and training strategies for segmenting nuclei from brightfield cell images. We tested U-Net as a baseline model; considered U-Net++, Tiramisu, and DeepLabv3+ as latest instances of advanced families of segmentation models; and propose PPU-Net, a novel light-weight alternative. The deeper architectures outperformed standard U-Net and results from previous studies on the challenging brightfield images, with balanced pixel-wise accuracies of up to 86%. PPU-Net achieved this performance with 20-fold fewer parameters than the comparably accurate methods. All models perform better on larger nuclei and in sparser images. We further confirmed that in the absence of plentiful training data, augmentation and pretraining on other data improve performance. In particular, using only 16 images with data augmentation is enough to achieve a pixel-wise F1 score that is within 5% of the one achieved with a full data set for all models. The remaining segmentation errors are mainly due to missed nuclei in dense regions, overlapping cells, and imaging artifacts, indicating the major outstanding challenges.

Live-cell microscopy or fluorescence anisotropy with budded baculoviruses—which way to go with measuring ligand binding to M 4 muscarinic receptors?

“…The focal plane had been manually chosen in a prior step. As it has been previously shown that similar DL network architectures require considerably more bright-field data to converge to an optimal solution compared to fluorescence data, a different strategy was chosen for training DL for cell detection from bright-field images [40,49,56]. As the training data volume was substantially larger, a data generator was used for cropping the images to the correct size (288 × 288 pixels) instead of predefined training and validation sets.…”

ArtSeg—Artifact segmentation and removal in brightfield cell microscopy images without manual pixel-level annotations

“…Overall, the datasets cover nine different cell lines, fixed and live cells, two different plate formats and two microscopes. The datasets provenances have been described previously 3 , 4 , 29 , 30 and we briefly describe their most important properties here.…”

“…However, the automated analysis techniques required to extract information at scale are often hindered by the artifacts present in the images 5 , 6 . Detecting and neutralizing the impact of such problematic image areas would provide more accurate results from experiments 3 , making artifact segmentation an important, albeit overlooked, research area in cell biology and beyond 7 , 8 .…”

A deep learning-based segmentation pipeline for profiling cellular morphodynamics using multiple types of live cell microscopy

“…A U-Net ( Ronneberger et al, 2015 ) is a CNN-based structure comprising an encoder, a decoder, and skip connections in between for segmentation. The architectural improvements of a U-Net-based structure yield even greater segmentation accuracy on microscopy images of cells or nuclei ( Ali et al, 2021 ; Caicedo et al, 2019 ; Raza, 2019 ). For instance, U-Net-based models such as StarDist ( Schmidt et al, 2018 ) and CellPose ( Stringer et al, 2021 ) have additional structures or outputs to segment images of crowded cells and nuclei effectively.…”