DeLTA: Automated cell segmentation, tracking, and lineage reconstruction using deep learning

Abstract

Microscopy image analysis is a major bottleneck in quantification of single-cell microscopy data, typically requiring human oversight and curation, which limit both accuracy and throughput. To address this, we developed a deep learning-based image analysis pipeline that performs segmentation, tracking, and lineage reconstruction. Our analysis focuses on time-lapse movies of Escherichia coli cells trapped in a "mother machine" microfluidic device, a scalable platform for long-term single-cell analysis that is widely used in the field. While deep learning has been applied to cell segmentation problems before, our approach is fundamentally innovative in that it also uses machine learning to perform cell tracking and lineage reconstruction. With this framework we are able to get high fidelity results (1% error rate), without human intervention. Further, the algorithm is fast, with complete analysis of a typical frame containing~150 cells taking <700msec. The framework is not constrained to a particular experimental set up and has the potential to generalize to time-lapse images of other organisms or different experimental configurations. These advances open the door to a myriad of applications including real-time tracking of gene expression and high throughput analysis of strain libraries at single-cell resolution.

UnMICST: Deep learning with real augmentation for robust segmentation of highly multiplexed images of human tissues

“…A variety of neural net and other architectures have been explored to improve detection and classification accuracy and expand generalizability to new types of images. Several deep learning architectures developed for natural images have been adapted for marker detection in images of cells including Fully Convolutional Networks (FCNs) (Lux and Matula, 2020), Visual Geometry Group (VGG16) (Wang et al, 2019;Shahzad M et al, 2020), Residual Networks (ResNets) (Lee and Jeong, 2020), UNet (Al-Kofahi et al, 2018;McQuin et al, 2018;Schmidt et al, 2018;Wen et al, 2018;Vu et al, 2019;Horwath et al, 2020;Lugagne, Lin and Dunlop, 2020), and Mask R-CNN (Kromp et al, 2019;Vuola, Akram andKannala, 2019, 2019;Korfhage et al, 2020;Liu et al, 2020;Masubuchi et al, 2020). In classical image analysis, advances in methodology commonly involve the development of new algorithms; any changes in parameter settings needed to accommodate new data are regarded as project-specific details.…”

“…A few studies have also added elastic deformations using B-splines (Ronneberger, Fischer and Brox, 2015;Raza et al, 2019;Torr et al, 2020). These methods are not unique to microscopy images, however, and only a few studies have used augmentation to address variation in the brightness and contrast of otherwise identical images (Lugagne, Lin and Dunlop, 2020) or added synthetically generated camera noise and non-cellular debris to make model training less sensitive to artefacts (Schmidt et al, 2018;Yang et al, 2020). A particularly interesting form of augmentation used by Kromp et al (2019) involved manually separating cells from the background and arranging nuclei in grids with random positions and orientations, effectively generating new training examples.…”

An updated toolkit for exploring bacterial cell wall structure and dynamics

“…However, automated microscopy experiments can generate immense data sets and can create a burden for image processing. To overcome this bottleneck, deep learning software has been introduced 79 .…”

Automated in vivo Tracking of Cortical Oligodendrocytes

“…Currently, most available cell tracking algorithms are designed for in vitro analysis and are not readily adaptable to in vivo conditions ( van Valen et al, 2016 ; Zhong et al, 2016 ; Nketia et al, 2017 ; Lugagne et al, 2020 ; Wang et al, 2020 ). The few in vivo tracking algorithms that exist are modality specific and cannot be readily adapted to our fluorescent longitudinal datasets ( Acton et al, 2002 ; Nguyen et al, 2011 ; Wang et al, 2015 ).…”