3DeeCellTracker, a deep learning-based pipeline for segmenting and tracking cells in 3D time lapse images

Abstract

Despite recent improvements in microscope technologies, segmenting and tracking cells in three-dimensional time-lapse images (3D + T images) to extract their dynamic positions and activities remains a considerable bottleneck in the field. We developed a deep learning-based software pipeline, 3DeeCellTracker, by integrating multiple existing and new techniques including deep learning for tracking. With only one volume of training data, one initial correction, and a few parameter changes, 3DeeCellTracker successfully segmented and tracked ~100 cells in both semi-immobilized and ‘straightened’ freely moving worm's brain, in a naturally beating zebrafish heart, and ~1000 cells in a 3D cultured tumor spheroid. While these datasets were imaged with highly divergent optical systems, our method tracked 90–100% of the cells in most cases, which is comparable or superior to previous results. These results suggest that 3DeeCellTracker could pave the way for revealing dynamic cell activities in image datasets that have been difficult to analyze.

Deep learning for bioimage analysis in developmental biology

“…The model then infers on the remaining data and its predictions are validated by the user. Other methods, such as 3DeeCellTracker ( Wen et al, 2021 ), rely on simulations to build large training sets with less need for human intervention.…”

Avoiding a replication crisis in deep-learning-based bioimage analysis

“…Over the last few years, deep learning (DL) has increasingly become one of the gold standards for high-performance microscopy image analysis 1 , 2 . DL has been shown to perform a wide range of image analysis very efficiently, such as image classification 3 , 4 , object detection 5 , 6 , image segmentation 7 – 9 , image restoration 10 , 11 , super-resolution microscopy 10 , 12 – 15 , object tracking 16 , 17 , image registration 18 and the prediction of fluorescence images from label-free imaging modalities 19 .…”

Deep learning is widely applicable to phenotyping embryonic development and disease

“…However, CNNs have also been applied to more complex tasks, such as reconstruction of the entire mouse brain vasculature ( Kirst et al, 2020 ; Todorov et al, 2020 ), in vivo quantification of cancer metastasis and in toto reconstruction of intact human organs ( Pan et al, 2019 ; Zhao et al, 2020 ). In embryology, deep learning has been used for Drosophila animal pose estimation, to map synaptic brain connections ( Buhmann et al, 2021 ; Graving et al, 2019 ; Günel et al, 2019 ), in C. elegans phenotyping ( Hakim et al, 2018 ; Saberi-Bosari et al, 2020 ) and in analysis of zebrafish beating hearts or vessels ( Akerberg et al, 2019 ; Kugler et al, 2020 preprint; Wen et al, 2021 ; Zhang et al, 2021 ), among other applications. Nevertheless, deep learning remains under-used in developmental biology ( Villoutreix, 2021 ).…”