Background: Tertiary lymphoid structures (TLS) are organized aggregates of immune cells that develop in non-lymphoid tissues and are associated with better prognosis and immunotherapy response across cancer types. Multiple IHC stainings are required for an accurate detection of TLS, making it challenging to implement as a clinical biomarker. Here, we developed a deep learning (DL) model that extracts nuclear morphology features to detect TLS from H&E slides and demonstrated its prognostic role in colorectal cancer (CRC) patients. Methods: A publicly available dataset consisting of 140 tissue cores from 35 CRC pts stained with H&E and 56 protein markers using the CODEX multiplex immunofluorescence (mIF) system was analyzed. Immune cell aggregates on the H&E were annotated by expert pathologists as either TLS or lymphocyte aggregates (LA), based on marker expression from the mIF stain on the same core. TLS were defined as dense aggregates of CD3+/CD20+/CD21+ cells, while all other immune cell aggregates were defined as LA. Next, HoVerNet was used to perform nuclear segmentation on cells within the TLS and LA on the H&E. Nuclear features including eccentricity, solidity, convexity, and nuclear intensity per cell were extracted and the mean and variance of each feature was summarized per tissue core. Based on these features, a univariate analysis comparing TLS and LA was performed, and a TLS classifier was trained using multivariate logistic regression. The classifier performance was assessed using 5 repeats of 5-fold cross validation and average accuracy and area under the ROC curve (AUC) were calculated. Overall survival (OS) was compared between patients with predicted TLS and LA using a Cox proportional hazard regression analysis. Results: From the 140 tissue cores, we identified cores with either TLS (n=18), LA (n=34) or none (n=92). No core presented both TLS and LA. In a Mann Whitney univariate analysis, cells in TLS areas demonstrated a higher mean nuclear eccentricity (p<0.0001) and solidity (p=0.01) along with lower variance in these features (p<0.0001 and p=0.001, respectively) compared to cells in LA. The multivariate classifier trained on nuclear features exhibited a 90.4% average accuracy (p<0.0001) and 94% AUC (p<0.0001) in differentiating between TLS and LA. Median OS was significantly higher in patients with at least one predicted TLS (n=13) vs. patients with at least one predicted LA (n=13) detected on H&E (NR vs. 19 months, HR=0.21, 95% CI 0.06-0.78; p=0.01). Conclusions: Nuclear based morphological features can be used to accurately detect the presence of TLS and LA from H&E slides, without the need for mIF or IHC stainings. Given the predictive value of TLS presence, this work demonstrates the potential for H&E slides to be used for patient selection for immunotherapy treatments. Citation Format: Becky Arbiv, Tal Dankovich, Sun Dagan, Yuval Shachaf, Tomer Dicker, Ron Elran, Avi Laniado, Amit Bart, Ori Zelichov, Ettai Markovits. Identification of tertiary lymphoid structures from H&E slides using deep learning analysis of nuclear morphology is associated with favorable survival in colorectal cancer patients. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 4316.
Background: Multiplex immunofluorescence (mIF) can provide invaluable insights into spatial biology and the complexities of the immune tumor microenvironment (iTME). However, existing analysis approaches are both laborious and highly user-dependent. In order to overcome these limitations we developed a novel, end-to-end deep learning (DL) pipeline for rapid and accurate analysis of both tumor-microarray (TMA) and whole slide mIF images. Methods: Our pipeline consists of two DL models: a multi-classifier for classifying multi-channel cell images into 12 different cell types, and a binary classifier for determining the positivity of a given marker in single-channel images. The DL multi-classifier was trained on 7,000 tiles labeled with cell annotations from a publicly available CODEX dataset, consisting of 140 tissue cores from 35 colorectal cancer (CRC) patients. For the binary classifier training, the multi-channel tiles were further split into ~100,000 single-channel tiles, for which the ground truth was inferred from the known expression of these markers in each cell-type. This DL binary classifier was then utilized to quantify the positivity of various cell state (phenotypic) markers. In addition, the binary classifier was exploited as a cell-typing tool, by predicting the positivity of individual lineage cell markers. The performance of our DL models was evaluated on 1,800 annotations from 14 test tissue cores. The models were further evaluated on a new 6-plex melanoma cohort, stained with PhenoImager, and were compared to the performance of clustering, manual thresholding or machine learning-based cell-typing methods applied on the same test sets. Results: Our DL multi-classifier achieved highly accurate results, outperforming all of the tested cell-typing methods, including clustering, manual-thresholding and ML-based approaches, in both CODEX CRC and PhenoImager melanoma cohorts (accuracy of 91% and 87%, respectively), with F1-scores above 80% in the vast majority of cell types. Our DL binary classifier, which was trained solely on the lineage markers of the CRC dataset, also outperformed existing methods, demonstrating excellent F1-scores (>80%) for determining the positivity of unseen phenotypic and lineage markers across the two tumor types and imaging modalities. Notably, as little as 20 annotations were required in order to boost the performance on an unseen dataset to above 85% accuracy and 80% F1-scores. As a result, the DL binary classifier could successfully be used as a cell-typing model, in a manner that is transferable between experimental approaches. Conclusions: We present a novel state-of-the-art DL-based framework for multiplex imaging analysis, that enables accurate cell typing and phenotypic marker quantification, which is robust across markers, tumor indications, and imaging modalities.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.