BCM3D 2.0: accurate segmentation of single bacterial cells in dense biofilms using computationally generated intermediate image representations

Abstract

Accurate detection and segmentation of single cells in three-dimensional (3D) fluorescence time-lapse images is essential for observing individual cell behaviors in large bacterial communities called biofilms. Recent progress in machine-learning-based image analysis is providing this capability with ever-increasing accuracy. Leveraging the capabilities of deep convolutional neural networks (CNNs), we recently developed bacterial cell morphometry in 3D (BCM3D), an integrated image analysis pipeline that combines deep learning with conventional image analysis to detect and segment single biofilm-dwelling cells in 3D fluorescence images. While the first release of BCM3D (BCM3D 1.0) achieved state-of-the-art 3D bacterial cell segmentation accuracies, low signal-to-background ratios (SBRs) and images of very dense biofilms remained challenging. Here, we present BCM3D 2.0 to address this challenge. BCM3D 2.0 is entirely complementary to the approach utilized in BCM3D 1.0. Instead of training CNNs to perform voxel classification, we trained CNNs to translate 3D fluorescence images into intermediate 3D image representations that are, when combined appropriately, more amenable to conventional mathematical image processing than a single experimental image. Using this approach, improved segmentation results are obtained even for very low SBRs and/or high cell density biofilm images. The improved cell segmentation accuracies in turn enable improved accuracies of tracking individual cells through 3D space and time. This capability opens the door to investigating time-dependent phenomena in bacterial biofilms at the cellular level.

Single‐cell segmentation in bacterial biofilms with an optimized deep learning method enables tracking of cell lineages and measurements of growth rates

“…While some deep learning methods rely on classical algorithms, for example, connected components and watershed, to separate individual touching cells in an instance segmentation task (Zhang et al, 2020(Zhang et al, , 2022, deep learning methods also allow developing of new instance segmentation methods that have an intrinsic representation for individual objects (Cutler et al, 2022;Schmidt et al, 2018;Stringer et al, 2021;Weigert et al, 2020). In these methods, boundaries between two cells are not enhanced for better postprocessing; rather the methods directly predict two individual cells.…”

“…The set of available sub‐volumes was then divided into sub‐volumes used for training, validation, and testing. We tested four CNN architectures for the final cell segmentation, StarDist (Weigert et al, 2020), Cellpose (Stringer et al, 2021), a multi‐class U‐Net (Zhang et al, 2020), and BCM3D 2.0 (Zhang et al, 2022).…”

“…We quantitatively evaluated the segmentation performance of StarDist OPP relative to the basic StarDist network, and different CNNs that were recently developed for bacterial cell segmentation: Cellpose (Stringer et al, 2021), a multi-class U-Net (Zhang et al, 2020), and BCM3D 2.0 (Zhang et al, 2022). All deep learning methods in this comparison were trained with the same ground truth dataset.…”

“…Using this dataset, we determined the minimum number of 3D annotated cells for obtaining accurate deep learning segmentations. We also used this large training dataset to evaluate the performance of recent deep learning algorithms (Schmidt et al, 2018;Stringer et al, 2021;Weigert et al, 2020;Zhang et al, 2020Zhang et al, , 2022 for single-cell segmentation in bacterial biofilms. For the well-performing StarDist algorithm, we developed a post-processing algorithm that optimizes the cell segmentation in biofilms, which outperforms the previous state-of-the-art segmentation methods for biofilms.…”

“…The set of available sub-volumes was then divided into subvolumes used for training, validation, and testing. We tested four CNN architectures for the final cell segmentation, StarDist(Weigert et al, 2020), Cellpose(Stringer et al, 2021), a multi-class U-Net(Zhang et al, 2020), and BCM3D 2.0(Zhang et al, 2022).The StarDist segmentation accuracy depends on the ray number to approximate the shape of the star-convex objects. By modeling all labels in the validation data as star-convex objects and comparing the model accuracy with respect to the input labels, we observed that 192 rays are sufficient to achieve a mean IoU value of 0.8 for the used training data in the anisotropic training data regime (FigureS12a).…”