Predicting Lymph Node Metastasis Using Histopathological Images Based on Multiple Instance Learning With Deep Graph Convolution

Zhao, Yu; Yang, Fan; Fang, Yuqi; Liu, Hailing; Zhou, Niyun; Zhang, Jun; Sun, Jiarui; Yang, Sen; Menze, Bjoern; Fan, Xinjuan; Yao, Jianhua

doi:10.1109/cvpr42600.2020.00489

Cited by 132 publications

(76 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, DL methods have been applied to segment and classify glomeruli with different staining and various pathologic changes, thus achieving the automatic analysis of renal biopsies ( 13 , 14 ); meanwhile, DL-based automatic colonoscopy tissue segmentation and classification have shown promise for colorectal cancer detection ( 15 , 16 ); besides, the analysis of gastric carcinoma and precancerous status can also benefit from DL schemes ( 17 , 18 ). More recently, for the ALN metastasis detection, it is reported that DL algorithms on digital lymph node pathology images achieved better diagnostic efficiency of ALN metastasis than pathologists ( 19 , 20 ). In particular, the assistance of algorithm significantly increases the sensitivity of detection for ALN micro-metastases ( 21 ).…”

Section: Introductionmentioning

confidence: 99%

“…In particular, the assistance of algorithm significantly increases the sensitivity of detection for ALN micro-metastases ( 21 ). In addition to diagnosis, several previous studies indicated that deep features based on whole-slide images (WSIs) of postoperative tumor samples potentially improved the prediction performance of lymph node metastasis in a variety of cancers ( 20 , 22 ). So far, there is no relevant research on preoperatively predicting ALN metastasis based on WSIs of primary BC samples.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Predicting Axillary Lymph Node Metastasis in Early Breast Cancer Using Deep Learning on Primary Tumor Biopsy Slides

Zhu

Tang

et al. 2021

Front. Oncol.

View full text Add to dashboard Cite

ObjectivesTo develop and validate a deep learning (DL)-based primary tumor biopsy signature for predicting axillary lymph node (ALN) metastasis preoperatively in early breast cancer (EBC) patients with clinically negative ALN.MethodsA total of 1,058 EBC patients with pathologically confirmed ALN status were enrolled from May 2010 to August 2020. A DL core-needle biopsy (DL-CNB) model was built on the attention-based multiple instance-learning (AMIL) framework to predict ALN status utilizing the DL features, which were extracted from the cancer areas of digitized whole-slide images (WSIs) of breast CNB specimens annotated by two pathologists. Accuracy, sensitivity, specificity, receiver operating characteristic (ROC) curves, and areas under the ROC curve (AUCs) were analyzed to evaluate our model.ResultsThe best-performing DL-CNB model with VGG16_BN as the feature extractor achieved an AUC of 0.816 (95% confidence interval (CI): 0.758, 0.865) in predicting positive ALN metastasis in the independent test cohort. Furthermore, our model incorporating the clinical data, which was called DL-CNB+C, yielded the best accuracy of 0.831 (95%CI: 0.775, 0.878), especially for patients younger than 50 years (AUC: 0.918, 95%CI: 0.825, 0.971). The interpretation of DL-CNB model showed that the top signatures most predictive of ALN metastasis were characterized by the nucleus features including density (p = 0.015), circumference (p = 0.009), circularity (p = 0.010), and orientation (p = 0.012).ConclusionOur study provides a novel DL-based biomarker on primary tumor CNB slides to predict the metastatic status of ALN preoperatively for patients with EBC.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Predicting Axillary Lymph Node Metastasis in Early Breast Cancer Using Deep Learning on Primary Tumor Biopsy Slides

Zhu

Tang

et al. 2021

Front. Oncol.

View full text Add to dashboard Cite

show abstract

“…In a follow-up work [25], they use a modified loss function which transfers labels to sliding windows based on the image-level label. Several studies propose using multi instance learning methods for histopathological image analysis [26][27][28][29]. Some of them also investigate the fusion of histopathology images with genomic or molecular data.…”

Section: Related Workmentioning

confidence: 99%

Deep Multi-Instance Learning Using Multi-Modal Data for Diagnosis of Lymphocytosis

Sahasrabudhe

Sujobert

Zacharaki

et al. 2021

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

We investigate the use of recent advances in deep learning and propose an end-to-end trainable multi-instance convolutional neural network within a mixture-of-experts formulation that combines information from two types of data-images and clinical attributes-for the diagnosis of lymphocytosis. The convolutional network learns to extract meaningful features from images of blood cells using an embedding level approach and aggregates them. Moreover, the mixture-of-experts model combines information from these images as well as clinical attributes to form an end-to-end trainable pipeline for diagnosis of lymphocytosis. Our results demonstrate that even the convolutional network by itself is able to discover meaningful associations between the images and the diagnosis, indicating the presence of important unexploited information in the images. The mixture-of-experts formulation is shown to be more robust while maintaining performance via. a repeatability study to assess the effect of variability in data acquisition on the predictions. The proposed methods are compared with different methods from literature based both on conventional handcrafted features and machine learning, and on recent deep learning models based on attention mechanisms. Our method reports a balanced accuracy of 85.41% and outperfroms the handcrafted feature-based and attention-based approaches as well that of biologists which scored 79.44%, 82.89% and 77.07% respectively. These results give insights on the potentials of the applicability of the proposed method in clinical practice. Our code and datasets can be found at https://www.github.com/msahasrabudhe/lymphoMIL.

show abstract

“…For example, DL methods have been applied to segment and classify glomeruli with different staining and various pathologic changes, thus achieving the automatic analysis of renal biopsies (13, 14); meanwhile, there has shown promise for colorectal cancer detection (15, 16) by DL based automatic colonoscopy tissue segmentation and classification; besides, the analysis of gastric carcinoma and precancerous status can also benefit from DL schemes (17, 18). More recently, for the ALN metastasis detection, it is reported that DL algorithms on digital lymph node pathology images achieved better diagnostic efficiency of ALN metastasis than pathologists (19, 20). In particular, the assistance of algorithm significantly increases the sensitivity of detection for ALN micro-metastases (21).…”

Section: Introductionmentioning

confidence: 99%

“…In particular, the assistance of algorithm significantly increases the sensitivity of detection for ALN micro-metastases (21). In addition to diagnosis, several previous studies indicated that deep features based on whole slide images (WSIs) of postoperative tumor samples potentially improved the prediction performance of lymph node metastasis in a variety of cancers (20, 22). So far, there is no relevant research on preoperatively predicting ALN metastasis based on WSIs of primary breast cancer samples.…”

Section: Introductionmentioning

confidence: 99%

Predicting Axillary Lymph Node Metastasis in Early Breast Cancer Using Deep Learning on Primary Tumor Biopsy Slides

Zhu

Tang

et al. 2021

Preprint

View full text Add to dashboard Cite

Objectives: To develop and validate a deep learning (DL) based primary tumor biopsy signature for predicting axillary lymph node (ALN) metastasis preoperatively in early breast cancer (EBC) patients with clinically negative ALN. Methods: A total of 1058 EBC patients with pathologically confirmed ALN status were enrolled from May 2010 to August 2020. A deep learning core-needle biopsy (DL-CNB) model was built on the attention based multiple instance learning (AMIL) framework to predict ALN status utilizing the deep learning features, which were extracted from the cancer areas of digitized whole-slide images (WSIs) of breast CNB specimens annotated by two pathologists. Accuracy, sensitivity, specificity, receiver operating characteristic (ROC) curves, and areas under the receiver operating characteristic curve (AUCs) were analyzed to evaluate our model. Results: The best performing DL-CNB model with VGG16_BN as the feature extractor achieved an AUC of 0.816 (95% confidence interval (CI): 0.758, 0.865) in predicting positive ALN metastasis in the independent test cohort. Furthermore, our model incorporating the clinical data, which was called DL-CNB+C, yielded the best accuracy of 0.831 (95%CI: 0.775, 0.878), especially for patients younger than 50 years (AUC: 0.918, 95%CI: 0.825, 0.971). The interpretation of DL-CNB model showed that the top signatures most predictive of ALN metastasis were characterized by the nuclei features including density (p=0.015), circumference (p=0.009), circularity (p=0.010), and orientation (p=0.012). Conclusion: Our study provides a novel deep learning-based biomarker on primary tumor CNB slides to predict the metastatic status of ALN preoperatively for patients with early breast cancer.

show abstract

Predicting Lymph Node Metastasis Using Histopathological Images Based on Multiple Instance Learning With Deep Graph Convolution

Cited by 132 publications

References 32 publications

Predicting Axillary Lymph Node Metastasis in Early Breast Cancer Using Deep Learning on Primary Tumor Biopsy Slides

Predicting Axillary Lymph Node Metastasis in Early Breast Cancer Using Deep Learning on Primary Tumor Biopsy Slides

Deep Multi-Instance Learning Using Multi-Modal Data for Diagnosis of Lymphocytosis

Predicting Axillary Lymph Node Metastasis in Early Breast Cancer Using Deep Learning on Primary Tumor Biopsy Slides

Contact Info

Product

Resources

About