Transformer-based unsupervised contrastive learning for histopathological image classification

Wang, Xiyue; Yang, Sen; Zhang, Jun; Wang, Minghui; Zhang, Jing; Yang, Wei; Huang, Junzhou; Han, Xiao

doi:10.1016/j.media.2022.102559

Cited by 156 publications

(112 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They showed that their histopathology-specific model outperformed a general-purpose contrastive learning self-supervised model (i.e., MoCo [ 58 ]) on three datasets—tumor metastasis detection, tissue type classification, and tumor cellularity quantification—under annotation-limited settings. Lastly, Wang et al developed a self-supervised method combined with self-attention to learn the patch-level embeddings [ 62 , 63 ], and then performed slide-level image retrieval based on said embeddings [ 64 ].…”

Section: Related Workmentioning

confidence: 99%

Contrastive Multiple Instance Learning: An Unsupervised Framework for Learning Slide-Level Representations of Whole Slide Histopathology Images without Labels

Tavolara

Gürcan

Niazi

2022

Cancers

View full text Add to dashboard Cite

Recent methods in computational pathology have trended towards semi- and weakly-supervised methods requiring only slide-level labels. Yet, even slide-level labels may be absent or irrelevant to the application of interest, such as in clinical trials. Hence, we present a fully unsupervised method to learn meaningful, compact representations of WSIs. Our method initially trains a tile-wise encoder using SimCLR, from which subsets of tile-wise embeddings are extracted and fused via an attention-based multiple-instance learning framework to yield slide-level representations. The resulting set of intra-slide-level and inter-slide-level embeddings are attracted and repelled via contrastive loss, respectively. This resulted in slide-level representations with self-supervision. We applied our method to two tasks— (1) non-small cell lung cancer subtyping (NSCLC) as a classification prototype and (2) breast cancer proliferation scoring (TUPAC16) as a regression prototype—and achieved an AUC of 0.8641 ± 0.0115 and correlation (R2) of 0.5740 ± 0.0970, respectively. Ablation experiments demonstrate that the resulting unsupervised slide-level feature space can be fine-tuned with small datasets for both tasks. Overall, our method approaches computational pathology in a novel manner, where meaningful features can be learned from whole-slide images without the need for annotations of slide-level labels. The proposed method stands to benefit computational pathology, as it theoretically enables researchers to benefit from completely unlabeled whole-slide images.

show abstract

Section: Related Workmentioning

confidence: 99%

Contrastive Multiple Instance Learning: An Unsupervised Framework for Learning Slide-Level Representations of Whole Slide Histopathology Images without Labels

Tavolara

Gürcan

Niazi

2022

Cancers

View full text Add to dashboard Cite

show abstract

“…In our case, various quality factors affect our performance, which include inconsistent lighting conditions or stain quality, stain differences and generalizing across sources [ 70 , 71 ]. State-of-the-art approaches have ushered in techniques for much more complicated classification tasks, including Gleason scoring (GS) or ISUP Gleason grading of prostate pathology [ 72 , 73 , 74 , 75 , 76 ]. Classifying indolent from cancer grade based on H&E pathology with multiple glands is empirically a difficult problem—there is a subtle distinction between any neighboring patterns with a fuzzy discrimination boundary between the pattern scoring levels [ 8 , 77 ].…”

Section: Discussionmentioning

confidence: 99%

Classifying Malignancy in Prostate Glandular Structures from Biopsy Scans with Deep Learning

Fogarty

Goldgof

Hall

et al. 2023

Cancers

View full text Add to dashboard Cite

Histopathological classification in prostate cancer remains a challenge with high dependence on the expert practitioner. We develop a deep learning (DL) model to identify the most prominent Gleason pattern in a highly curated data cohort and validate it on an independent dataset. The histology images are partitioned in tiles (14,509) and are curated by an expert to identify individual glandular structures with assigned primary Gleason pattern grades. We use transfer learning and fine-tuning approaches to compare several deep neural network architectures that are trained on a corpus of camera images (ImageNet) and tuned with histology examples to be context appropriate for histopathological discrimination with small samples. In our study, the best DL network is able to discriminate cancer grade (GS3/4) from benign with an accuracy of 91%, F1-score of 0.91 and AUC 0.96 in a baseline test (52 patients), while the cancer grade discrimination of the GS3 from GS4 had an accuracy of 68% and AUC of 0.71 (40 patients).

show abstract

“…Computer algorithms have been in use since 1993 for image analysis for cancer grading, diagnosis and prognostic predictions [18]. Recently, deep learning based algorithms have dramatically improved the results of image analysis of pathological images [19][20][21]. Many DL based model have been developed to use for histopathological image analysis in lung cancer.…”

Section: Discussionmentioning

confidence: 99%

DeepMPS: Development and validation of a deep learning model for whole slide image base prognostic prediction of low grade Lung adenocarcinoma patients

Jagajampi

Khadirnaikar

Malik

et al. 2022

Preprint

View full text Add to dashboard Cite

Lung adenocarcinoma (LUAD) is one of the most common cancers, and patients' prognostication is crucial for treatment decisions. Histopathological images are the most generally accessible clinical information, however they have not been employed in clinical settings for prognosis. In this study, we used WSIs and clinical data from TCGA (training and testing) and East Asian Cohort (EAS, Validation) to develop and validate DL-based prognosticator. To circumvent the need for manual ROI generation, WSIs from these patients were divided into smaller patches and scored. DeepMPS prediction model was built using the top scoring 226,383 patches. The DeepMPS model showed a C-index of 0.638 in the TCGA training cohort. The univariate and multivariate cox regression analysis identified DeepMPS as an independent predictor of survival (HR: 9.48, p-value: <0.0001) in the training cohort. The training cohort of patients was separated into low and high-risk groups at various points. Kaplan-Meier analysis showed the highest difference in survival of low and high-risk patients at the 75th percentile (HR: 3.58, 95% CI: 2.57-5.00, p-value: <0.0001). At the same cut-off as the training samples, TCGA testing cohort patients demonstrated a significant difference in survival when split into low and high-risk patients (HR: 2.30, 95% CI: 1.11-4.82, p-value: 0.044). The DeepMPS model was validated in the EAS cohort patients. DeepMPS risk score significantly segregated EAS patients into low and high-risk groups at the same cut-off point as the training cohort (HR: 2.09, 95% CI: 1.11-3.97, p-value: 0.008). In multivariate Cox regression analysis, the DeepMPS score outperformed the stage in survival prediction. We also compared the DeepMPS model with the previously developed DL-based model to show that it was the best predictor of survival with the highest C-index. In conclusion, we developed a robust DL-based prognostic model which can predict the LUAD outcome without manual intervention using histopathological images.

show abstract

Transformer-based unsupervised contrastive learning for histopathological image classification

Cited by 156 publications

References 32 publications

Contrastive Multiple Instance Learning: An Unsupervised Framework for Learning Slide-Level Representations of Whole Slide Histopathology Images without Labels

Contrastive Multiple Instance Learning: An Unsupervised Framework for Learning Slide-Level Representations of Whole Slide Histopathology Images without Labels

Classifying Malignancy in Prostate Glandular Structures from Biopsy Scans with Deep Learning

DeepMPS: Development and validation of a deep learning model for whole slide image base prognostic prediction of low grade Lung adenocarcinoma patients

Contact Info

Product

Resources

About