RetCCL: Clustering-guided contrastive learning for whole-slide image retrieval

Wang, Xiyue; Du, Yuexi; Yang, Sen; Zhang, Jun; Wang, Minghui; Zhang, Jing; Yang, Wei; Huang, Junzhou; Han, Xiao

doi:10.1016/j.media.2022.102645

Cited by 89 publications

(72 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We chose this model due to its broad use in the computational pathology research literature 14 . The second model is the Retrieval with Clustering-guided Contrastive Learning (RetCCL) 32 model, a Resnet50 backbone that was trained on a pathology dataset with Self Supervised Learning (SSL).…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Self-supervised attention-based deep learning for pan-cancer mutation prediction from histopathology

et al. 2023

View full text Add to dashboard Cite

The histopathological phenotype of tumors reflects the underlying genetic makeup. Deep learning can predict genetic alterations from pathology slides, but it is unclear how well these predictions generalize to external datasets. We performed a systematic study on Deep-Learning-based prediction of genetic alterations from histology, using two large datasets of multiple tumor types. We show that an analysis pipeline that integrates self-supervised feature extraction and attention-based multiple instance learning achieves a robust predictability and generalizability.

show abstract

Section: Methodsmentioning

confidence: 99%

“…8). We then used our in-house open-source DL pipeline (https:// github.com/KatherLab/marugoto) which uses the SSL-trained model RetCCL 32 to obtain 2048 features per tile and uses attMIL to make patient-level predictions 33,34 .…”

Section: Data Acquisition and Experimental Designmentioning

confidence: 99%

Self-supervised attention-based deep learning for pan-cancer mutation prediction from histopathology

et al. 2023

View full text Add to dashboard Cite

show abstract

“…They showed that their histopathology-specific model outperformed a general-purpose contrastive learning self-supervised model (i.e., MoCo [ 58 ]) on three datasets—tumor metastasis detection, tissue type classification, and tumor cellularity quantification—under annotation-limited settings. Lastly, Wang et al developed a self-supervised method combined with self-attention to learn the patch-level embeddings [ 62 , 63 ], and then performed slide-level image retrieval based on said embeddings [ 64 ].…”

Section: Related Workmentioning

confidence: 99%

Contrastive Multiple Instance Learning: An Unsupervised Framework for Learning Slide-Level Representations of Whole Slide Histopathology Images without Labels

Tavolara

Gürcan

Niazi

2022

Cancers

View full text Add to dashboard Cite

Recent methods in computational pathology have trended towards semi- and weakly-supervised methods requiring only slide-level labels. Yet, even slide-level labels may be absent or irrelevant to the application of interest, such as in clinical trials. Hence, we present a fully unsupervised method to learn meaningful, compact representations of WSIs. Our method initially trains a tile-wise encoder using SimCLR, from which subsets of tile-wise embeddings are extracted and fused via an attention-based multiple-instance learning framework to yield slide-level representations. The resulting set of intra-slide-level and inter-slide-level embeddings are attracted and repelled via contrastive loss, respectively. This resulted in slide-level representations with self-supervision. We applied our method to two tasks— (1) non-small cell lung cancer subtyping (NSCLC) as a classification prototype and (2) breast cancer proliferation scoring (TUPAC16) as a regression prototype—and achieved an AUC of 0.8641 ± 0.0115 and correlation (R2) of 0.5740 ± 0.0970, respectively. Ablation experiments demonstrate that the resulting unsupervised slide-level feature space can be fine-tuned with small datasets for both tasks. Overall, our method approaches computational pathology in a novel manner, where meaningful features can be learned from whole-slide images without the need for annotations of slide-level labels. The proposed method stands to benefit computational pathology, as it theoretically enables researchers to benefit from completely unlabeled whole-slide images.

show abstract

“…Secondly, the patches for each cohort were color normalized using the Macenko spectral matching technique (40) to enforce a standardized color distribution across cohorts. To train the prediction models, we used our in-house open-source DL pipeline "marugoto" (https://github.com/KatherLab/marugoto) consisting of a self-supervised learning (SSL) model using a pre-trained ResNet50 architecture with ImageNet weights, fine-tuned pan-cancer on approximately 32.000 WSI to extract a 2048-dimensional feature vector for each patch per patient (41). To obtain patient-level predictions, 512x2048 feature matrices (MIL bags) are constructed by concatenating 512 feature vectors selected at random per patient and fed into an attMIL framework with the following architecture: (512x256), (256x2) with a subsequent attention mechanism (Figure 1B).…”

Section: Image Preprocessingmentioning

confidence: 99%

Direct prediction of Homologous Recombination Deficiency from routine histology in ten different tumor types with attention-based Multiple Instance Learning: a development and validation study

Loeffler

Nahhas

Muti

et al. 2023

Preprint

View full text Add to dashboard Cite

Background: Homologous Recombination Deficiency (HRD) is a pan-cancer predictive biomarker that identifies patients who benefit from therapy with PARP inhibitors (PARPi). However, testing for HRD is highly complex. Here, we investigated whether Deep Learning can predict HRD status solely based on routine Hematoxylin & Eosin (H&E) histology images in ten cancer types. Methods: We developed a fully automated deep learning pipeline with attention-weighted multiple instance learning (attMIL) to predict HRD status from histology images. A combined genomic scar HRD score, which integrated loss of heterozygosity (LOH), telomeric allelic imbalance (TAI) and large-scale state transitions (LST) was calculated from whole genome sequencing data for n=4,565 patients from two independent cohorts. The primary statistical endpoint was the Area Under the Receiver Operating Characteristic curve (AUROC) for the prediction of genomic scar HRD with a clinically used cutoff value. Results: We found that HRD status is predictable in tumors of the endometrium, pancreas and lung, reaching cross-validated AUROCs of 0.79, 0.58 and 0.66. Predictions generalized well to an external cohort with AUROCs of 0.93, 0.81 and 0.73 respectively. Additionally, an HRD classifier trained on breast cancer yielded an AUROC of 0.78 in internal validation and was able to predict HRD in endometrial, prostate and pancreatic cancer with AUROCs of 0.87, 0.84 and 0.67 indicating a shared HRD-like phenotype is across tumor entities. Conclusion: In this study, we show that HRD is directly predictable from H&E slides using attMIL within and across ten different tumor types.

show abstract

RetCCL: Clustering-guided contrastive learning for whole-slide image retrieval

Cited by 89 publications

References 35 publications

Self-supervised attention-based deep learning for pan-cancer mutation prediction from histopathology

Self-supervised attention-based deep learning for pan-cancer mutation prediction from histopathology

Contrastive Multiple Instance Learning: An Unsupervised Framework for Learning Slide-Level Representations of Whole Slide Histopathology Images without Labels

Direct prediction of Homologous Recombination Deficiency from routine histology in ten different tumor types with attention-based Multiple Instance Learning: a development and validation study

Contact Info

Product

Resources

About