Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning

Greenwald, Noah F.; Miller, Geneva; Moen, Erick; Kong, Alex; Kagel, Adam; Fullaway, Christine Camacho; McIntosh, Brianna J.; Leow, Ke Xuan; Schwartz, Morgan; Dougherty, Thomas; Pavelchek, Cole; Cui, Sunny; Camplisson, Isabella; Bar-Tal, Omer; Singh, Jaiveer; Fong, Mara; Chaudhry, Gautam; Abraham, Zion; Moseley, Jackson; Warshawsky, Shiri; Soon, Erin; Greenbaum, Shirley; Risom, Tyler; Hollmann, Travis J.; Keren, Leeat; Graf, William D.; Angelo, Michael; Valen, David Van

doi:10.1101/2021.03.01.431313

Cited by 70 publications

(83 citation statements)

References 76 publications

(142 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compared Dice-XMBD with a generic whole-cell segmentation method across six imaging platforms, Mesmer [41], which used a deep learning-based algorithm trained on a large, annotated image dataset to segment single cells and nuclei separately. A trained Mesmer model was tested with combined nuclear and cell channels which is the same as the input to Dice-XMBD.…”

Section: Resultsmentioning

confidence: 99%

Dice-XMBD: Deep learning-based cell segmentation for imaging mass cytometry

Qiao

Jiao

et al. 2021

Preprint

View full text Add to dashboard Cite

Highly multiplexed imaging technology is a powerful tool to facilitate understanding cells composition and interaction in tumor microenvironment at subcellular resolution, which is crucial for both basic research and clinical applications. Imaging mass cytometry (IMC), a multiplex imaging method recently introduced, can measure up to 40 markers simultaneously in one tissue section by using a high-resolution laser with a mass cytometer. However, due to its high resolution and large number of channels, how to process and interpret the image data from IMC remains a key challenge for its further applications. Accurate and reliable single cell segmentation is the first and a critical step to process IMC image data. Unfortunately, existing segmentation pipelines either produce inaccurate cell segmentation results, or require manual annotation which is very time-consuming. Here, we developed Dice-XMBD, a Deep learnIng-based Cell sEgmentation algorithm for tissue multiplexed imaging data. In comparison with other state-of-the-art cell segmentation methods currently used in IMC, Dice-XMBD generates more accurate single cell masks efficiently on IMC images produced with different nuclear, membrane and cytoplasm markers. All codes and datasets are available at https://github.com/xmuyulab/Dice-XMBD.

show abstract

Section: Resultsmentioning

confidence: 99%

Dice-XMBD: Deep learning-based cell segmentation for imaging mass cytometry

Qiao

Jiao

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…MIBI image cell segmentation was obtained with Mesmer, a deep learning algorithm based on the DeepCell library (deepcelltf 0.6.0) (Van Valen et al 2016; Greenwald et al 2021). The neural network weights for prediction were imported from https://deepcell-data.s3-us-west-1.amazonaws.com/model-weights/Multiplex_Segmentation_20200908_2_head.h5.…”

Section: Methodsmentioning

confidence: 99%

Spatial epitope barcoding reveals subclonal tumor patch behaviors

Rovira-Clavé

Drainas

Jiang

et al. 2021

Preprint

View full text Add to dashboard Cite

Intratumoral variability is a seminal feature of human tumors contributing to tumor progression and response to treatment. Current technologies are unsuitable to accurately track phenotypes and subclonal evolution within tumors, especially in response to genetic manipulations. Here, we developed epitope combinatorial tags (EpicTags), which we coupled to multiplexed ion beam imaging (EpicMIBI) for in situ tracking of barcodes within tissue microenvironments. Using this platform, we dissected the spatial component of cell lineages and phenotypes in a xenograft model of small-cell lung cancer. We observed emergent properties from mixed clones leading to the preferential expansion of subclonal patches for both neuroendocrine and non-neuroendocrine cancer cell states in this model. In tumors harboring a fraction of PTEN-deficient cancer cells, we uncovered a non-autonomous increase of subclonal patch size in PTEN wild-type cancer cells. EpicMIBI can facilitate in situ interrogation of cell-intrinsic and cell-extrinsic processes involved in intratumoral heterogeneity.

show abstract

“…Accurate cell segmentation methods are required to confidently extract single-cell feature information from multiplexed tissue images (Hollandi et al, 2020;Moen et al, 2019;Valen et al, 2016). We used the top-in-class Mesmer, a DeepCell-based segmentation method for feature extraction and FlowSOM for subsequent cell type identification utilizing self-organizing maps (Figure 2A) (Gassen et al, 2015;Greenwald et al, 2021). We identified 14 distinct immune and structural cell types, with the expected associated lineage-specific marker expression (Figure 2B).…”

Section: R a F Tmentioning

confidence: 99%

“…Image Segmentation. Cell segmentation was performed using a local implementation of Mesmer, which utilizes the Dep-pCell library (deepcell-tf 0.6.0) as described (Greenwald et al, 2021;Valen et al, 2016). We adapted the included multi-plex_segmentation.py python script from the deepcell-tf library and imported the neural network weights for prediction from https://deepcell-data.s3-us-west-1.amazonaws.com/model-weights/Multiplex_Segmentation_20200908_2_head.h5).…”

Section: Data Availabilitymentioning

confidence: 99%

Virus-Dependent Immune Conditioning of Tissue Microenvironments

Jiang

Chan

Rovira-Clavé

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

A thorough understanding of complex spatial host-disease interactions in situ is necessary in order to develop effective preventative measures and therapeutic strategies. Here, we developed Protein And Nucleic acid IN situ Imaging (PANINI) and coupled it with Multiplexed Ion Beam Imaging (MIBI) to sensitively and simultaneously quantify DNA, RNA, and protein levels within the microenvironments of tissue compartments. The PANINI-MIBI approach was used to measure over 30 parameters simultaneously across large sections of archival lymphoid tissues from non-human primates that were healthy or infected with simian immunodeficiency virus (SIV), a model that accurately recapitulates human immunodeficiency virus infection (HIV). This enabled multiplexed dissection of cellular phenotypes, functional markers, viral DNA integration events, and viral RNA transcripts as resulting from viral infection. The results demonstrated immune coordination from an unexpected upregulation of IL10 in B cells in response to SIV infection that correlated with macrophage M2 polarization, thus conditioning a potential immunosuppressive environment that allows for viral production. This multiplexed imaging strategy also allowed characterization of the coordinated microenvironment around latently or actively infected cells to provide mechanistic insights into the process of viral latency. The spatial multi-modal framework presented here is applicable to deciphering tissue responses in other infectious diseases and tumor biology.

show abstract

Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning

Cited by 70 publications

References 76 publications

Dice-XMBD: Deep learning-based cell segmentation for imaging mass cytometry

Dice-XMBD: Deep learning-based cell segmentation for imaging mass cytometry

Spatial epitope barcoding reveals subclonal tumor patch behaviors

Virus-Dependent Immune Conditioning of Tissue Microenvironments

Contact Info

Product

Resources

About