Lymph node metastasis examined by the resected lymph nodes is considered one of the most important prognostic factors for colorectal cancer (CRC). However, it requires careful and comprehensive inspection by expert pathologists. To relieve the pathologists’ burden and speed up the diagnostic process, in this paper, we develop a deep learning system with the binary positive/negative labels of the lymph nodes to solve the CRC lymph node classification task. The multi-instance learning (MIL) framework is adopted in our method to handle the whole slide images (WSIs) of gigapixels in size at once and get rid of the labor-intensive and time-consuming detailed annotations. First, a transformer-based MIL model, DT-DSMIL, is proposed in this paper based on the deformable transformer backbone and the dual-stream MIL (DSMIL) framework. The local-level image features are extracted and aggregated with the deformable transformer, and the global-level image features are obtained with the DSMIL aggregator. The final classification decision is made based on both the local and the global-level features. After the effectiveness of our proposed DT-DSMIL model is demonstrated by comparing its performance with its predecessors, a diagnostic system is developed to detect, crop, and finally identify the single lymph nodes within the slides based on the DT-DSMIL and the Faster R-CNN model. The developed diagnostic model is trained and tested on a clinically collected CRC lymph node metastasis dataset composed of 843 slides (864 metastasis lymph nodes and 1415 non-metastatic lymph nodes), achieving the accuracy of 95.3% and the area under the receiver operating characteristic curve (AUC) of 0.9762 (95% confidence interval [CI]: 0.9607–0.9891) for the single lymph node classification. As for the lymph nodes with micro-metastasis and macro-metastasis, our diagnostic system achieves the AUC of 0.9816 (95% CI: 0.9659–0.9935) and 0.9902 (95% CI: 0.9787–0.9983), respectively. Moreover, the system shows reliable diagnostic region localizing performance: the model can always identify the most likely metastases, no matter the model’s predictions or manual labels, showing great potential in avoiding false negatives and discovering incorrectly labeled slides in actual clinical use. Graphical Abstract
3543 Background: Immunotherapy has brought about a landmark change in anti-tumor treatment in the past years. High microsatellite instability (MSI-H) is now the only clinically approved biomarker predicting response to immunotherapy in CRC. Increasing evidence suggests that POLE mutations in the exonuclease domain could drive an ultra-mutational phenotype and improve the treatment outcomes of ICI in solid tumors. In this study, we set out to apply a deep learning model using H&E-stained, formalin-fixed, paraffin-embedded (FFPE) whole slide images (WSIs) of CRC primary tumors. Methods: The deep learning model is developed and validated through five-fold cross-validation using WSI of primary tumors from 506 CRC patients and externally validated using 52 WSIs from a prospective cohort. The microsatellite status, tumor mutation burden (TMB) and POLE genotype were determined by next-generation sequencing (NGS). Patients with MSS status and a low TMB (<20Mutations/Mb) were admitted to the MSS group, and CRCs with a POLE mutation which was defined as an oncogenic mutation referring to the POLE functional mutation list at OncoKB( POLE (oncokb.org) were admitted to the POLE mutant group. Clustering-constrained-attention multiple-instance learning (CLAM) model is employed as the base model, and we conduct the model ensemble by performing a large-scale hyper-parameter search, selecting five models with the highest value in one of the performance metrics, including the AURoC, accuracy, precision, recall, and f1 score, and finally averaging the predictions of the five models. Results: The internal dataset included 237 MSS, 142 MSI-H, and 127 POLE mutant CRC. The three groups had significant differences in primary location (p < 0.0001), histology (p < 0.0001), tumor differentiation (p = 0.002), tumor stage (p < 0.0001), Crohn's-like reaction (p < 0.0001) and tumor growth pattern (p = 0.001). The cross-validation performance of the ensemble model (M E) in the internal dataset achieves an AURoC of 0.944 for three-way classification task (POLE vs. MSI-H vs. MSS) and 0.940 for two-way classification task (POLE & MSI-H vs. MSS) which were superior to the performance of each single CLAM model. To demonstrate the generalizability of the deep learning model, a domestic perspective cohort consisting of 20 MSS, 17 MSI-H, and 15 POLE mutant CRC H&E images were used to validate the external performance. And the M E retained robust performance on the external dataset, with an AURoC of 0.904 for three-way classification task and 0.836 for two-way classification task. Conclusions: A CLAM-based deep learning model could directly predict the MSI-H and POLE mutation from histological images that could be used to stratify CRC patients for immunotherapy with faster turnaround time and lower costs compared with traditional sequencing methods.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.