Caliban: Accurate cell tracking and lineage construction in live-cell imaging experiments with deep learning

Abstract

Live-cell imaging experiments have opened an exciting window into the behavior of living systems. While these experiments can produce rich data, the computational analysis of these datasets is challenging. Single-cell analysis requires that cells be accurately identified in each image and subsequently tracked over time. Increasingly, deep learning is being used to interpret microscopy image with single cell resolution. In this work, we apply deep learning to the problem of tracking single cells in live-cell imaging data. Using crowdsourcing and a human-in-the-loop approach to data annotation, we constructed a dataset of over 11,000 trajectories of cell nuclei that includes lineage information. Using this dataset, we successfully trained a deep learning model to perform cell tracking within a linear programming framework. Benchmarking tests demonstrate that our method achieves state-of-the-art performance on the task of cell tracking with respect to multiple accuracy metrics. Further, we show that our deep learning-based method generalizes to perform cell tracking for both fluorescent and brightfield images of the cell cytoplasm, despite having never been trained on those data types. This enables analysis of live-cell imaging data collected across imaging modalities. A persistent cloud deployment of our cell tracker is available at http://www.deepcell.org.

Interpretable deep learning of label-free live cell images uncovers functional hallmarks of highly-metastatic melanoma

“…However, the often cited weakness of these techniques is the lack of an intuitive explanation of which parts of the data are particularly meaningful in defining the extracted pattern. While in some applications, such as image segmentation, image restoration or mapping between imaging modalities, a well-validated outcome of a network has been satisfactory (Christiansen et al, 2018;Fang et al, 2019b;Guo et al, 2019;Hershko et al, 2019;Hollandi et al, 2019;LaChance and Cohen, 2020;Moen et al, 2019;Nehme et al, 2018;Ounkomol et al, 2018;Ouyang et al, 2018;Rivenson et al, 2019;Wang et al, 2019;Weigert et al, 2018;Wu et al, 2019), there is increasing mistrust in results produced by 'black-box' neural networks. Aside from increasing the confidence, the analysis of the properties -also referred to as 'mechanisms'of the pattern recognition process can potentially generate insight of a biological/physical phenomenon that escapes the analysis driven by human intuition.…”

NuSeT: A deep learning tool for reliably separating and analyzing crowded cells

“…Since in biological image processing the primary focus is on cell-level segmentation rather than pixel-level accuracy, we also included object-level segmentation metrics, including the rate of correctly separating overlapping nuclei, correct and incorrect detections, splits, merges, catastrophes, and both the false-positive and false-negative detection rates (Methods). [ 29 , 30 ] Two separate datasets, ‘MCF10A’ and ‘Kaggle’, were used to compare the performance of the algorithms. [ 33 ] The MCF10A dataset consists of images of relatively uniformly fluorescent nuclei of a non-tumorigenic breast epithelial cell line[ 37 ], grown to different levels of confluence.…”

“…A more recent implementation of this approach replaces the RPN with a single shot detection module[ 27 ], achieving superior performance in segmenting and tracking cells and nuclei. [ 28 , 29 ] However, the performance of Mask R-CNN based approaches remains to be validated for images with high cell density. Mask R-CNN also employs fixed anchor scales for bounding boxes across all images, which is a limitation for samples with variable-sized nuclei.…”

Comparison of Cell and Organoid-Level Analysis of Patient-Derived 3D Organoids to Evaluate Tumor Cell Growth Dynamics and Drug Response

“…Moreover, implementing machine learning algorithms within our analysis pipeline to detect organoid features will significantly reduce image analysis time. 47,48 Patient organoids can behave differently (i.e., growth rates and drug effects) based on their genetic and environmental backgrounds. 49 We tested two different patient-derived organoids in this study and observed differential growth rates between them.…”