A self-supervised contrastive learning approach for whole slide image representation in digital pathology

Fashi, Parsa Ashrafi; Hemati, Sobhan; Babaie, Morteza; Gonzalez, Ricardo; Tizhoosh, Hamid R.

doi:10.1016/j.jpi.2022.100133

Cited by 17 publications

(13 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rather remarkably, they noted that neighboring WSI patches look the same and thus should have similar embeddings. Finally, Fashi et al utilized contrastive learning based on site-of-origin labels as pseudo-labels for pre-training, then applied attention pooling on the resulting embeddings to classify WSIs [ 40 ]. They astutely noted that site-of-origin is nearly always available and thus, should be incorporated into the self-supervised objective (i.e., as a label).…”

Section: Related Workmentioning

confidence: 99%

“…Another study by Li et al [ 24 ] utilized SimCLR [ 39 ] on WSI patches as a pre-training step for subsequent MIL-based classification. Fashi et al [ 40 ] utilized contrastive learning based on site-of-origin labels as pseudo-labels for pre-training then applied attention pooling on the resulting embeddings to classify WSIs. All studies demonstrated similar benefits at the slide level that were considered at the patch level.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Contrastive Multiple Instance Learning: An Unsupervised Framework for Learning Slide-Level Representations of Whole Slide Histopathology Images without Labels

Tavolara

Gürcan

Niazi

2022

Cancers

View full text Add to dashboard Cite

Recent methods in computational pathology have trended towards semi- and weakly-supervised methods requiring only slide-level labels. Yet, even slide-level labels may be absent or irrelevant to the application of interest, such as in clinical trials. Hence, we present a fully unsupervised method to learn meaningful, compact representations of WSIs. Our method initially trains a tile-wise encoder using SimCLR, from which subsets of tile-wise embeddings are extracted and fused via an attention-based multiple-instance learning framework to yield slide-level representations. The resulting set of intra-slide-level and inter-slide-level embeddings are attracted and repelled via contrastive loss, respectively. This resulted in slide-level representations with self-supervision. We applied our method to two tasks— (1) non-small cell lung cancer subtyping (NSCLC) as a classification prototype and (2) breast cancer proliferation scoring (TUPAC16) as a regression prototype—and achieved an AUC of 0.8641 ± 0.0115 and correlation (R2) of 0.5740 ± 0.0970, respectively. Ablation experiments demonstrate that the resulting unsupervised slide-level feature space can be fine-tuned with small datasets for both tasks. Overall, our method approaches computational pathology in a novel manner, where meaningful features can be learned from whole-slide images without the need for annotations of slide-level labels. The proposed method stands to benefit computational pathology, as it theoretically enables researchers to benefit from completely unlabeled whole-slide images.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Contrastive Multiple Instance Learning: An Unsupervised Framework for Learning Slide-Level Representations of Whole Slide Histopathology Images without Labels

Tavolara

Gürcan

Niazi

2022

Cancers

View full text Add to dashboard Cite

show abstract

“…Mesothelioma tissue WSIs were used to build classifier to identify transitional mesothelioma (TM) or not‐TM tissue 138,139 . Finally, research has studied tumor versus non‐tumor and TIL classification across different cancer types 33–35,140–143 …”

Section: Clinical Tasks In Computational Histopathologymentioning

confidence: 99%

Machine learning in computational histopathology: Challenges and opportunities

Cooper

Krishnan

2023

Genes Chromosomes & Cancer

View full text Add to dashboard Cite

Digital histopathological images, high-resolution images of stained tissue samples, are a vital tool for clinicians to diagnose and stage cancers. The visual analysis of patient state based on these images are an important part of oncology workflow. Although pathology workflows have historically been conducted in laboratories under a microscope, the increasing digitization of histopathological images has led to their analysis on computers in the clinic. The last decade has seen the emergence of machine learning, and deep learning in particular, a powerful set of tools for the analysis of histopathological images. Machine learning models trained on large datasets of digitized histopathology slides have resulted in automated models for prediction and stratification of patient risk. In this review, we provide context for the rise of such models in computational histopathology, highlight the clinical tasks they have found success in automating, discuss the various machine learning techniques that have been applied to this domain, and underscore open problems and opportunities.

show abstract

“…KimiaNet reported two types of image search: horizontal search and vertical search. In the horizontal search, the query is applied to the entire data set to find similar whole slide images (WSI) with a self-supervised model 27 , while in the study by Fashi et al 28 , the vertical search approach is designed to identify similar types of malignancies in a specific organ. This is achieved by utilizing pretrained models with openly provided weights from the Keras library.…”

Section: Content-based Medical Image Retrieval (Cbmir)mentioning

confidence: 99%

Towards More Transparent and Accurate Cancer Diagnosis with an Unsupervised CAE Approach

Tabatabaei¹,

Colomer²,

Moll³

et al. 2023

Preprint

View full text Add to dashboard Cite

Background: Digital pathology has significantly impacted the cancer diagnosis field, with Content-Based Medical Image Retrieval (CBMIR) emerging as a powerful tool for analyzing histopathological Whole Slide Images (WSIs). CBMIR allows users to search a database for similar content to a query, providing pathologists with access to collections of cases with comparable features. This can improve the reliability of diagnostic references and help in making more accurate and timely diagnoses. Objective: In 2020, the Global Cancer Observatory (GCO) reported that breast cancer is the most prevalent cancer type in both men and women, accounting for 11.7% of all cases, while prostate cancer is the second most common cancer type in men, comprising 14.1% of cases. The aim of the proposed Unsupervised CBMIR (UCBMIR) is to replicate the traditional cancer diagnosis workflow and provide a dependable method for supporting pathologists when making diagnostic conclusions based on WSIs. By reducing the workload of pathologists, this approach could potentially enhance the accuracy and efficiency of cancer diagnosis. Method and results: The study presents an innovative approach to address the problem of the lack of labeled histopathological images in CBMIR. A customized unsupervised Convolutional Auto Encoder (CAE) was developed to extract 200 features per image, which were then used by the search engine component. The proposed UCBMIR was evaluated using two widely used numerical techniques in CBMIR and visual evaluation, and compared with a classifier to determine if retrieved images belong to the same cancer type as the query. The validation process was conducted using three distinct data sets, with an external evaluation to demonstrate its effectiveness. The UCBMIR outperformed previous studies, achieving a top 5 recall of 99% and 80% on BreaKHis and SICAPv2, respectively, using the first evaluation technique. Using the second evaluation technique, UCBMIR achieved precision rates of 91% and 70% for BreaKHis and SICAPv2, respectively. Moreover, the UCBMIR was able to identify various patterns in patches and achieved an accuracy of 81% in the top 5 when tested on an external image from Arvaniti, having been trained using SICAPv2 with the second evaluation technique.

show abstract

A self-supervised contrastive learning approach for whole slide image representation in digital pathology

Cited by 17 publications

References 25 publications

Contrastive Multiple Instance Learning: An Unsupervised Framework for Learning Slide-Level Representations of Whole Slide Histopathology Images without Labels

Contrastive Multiple Instance Learning: An Unsupervised Framework for Learning Slide-Level Representations of Whole Slide Histopathology Images without Labels

Machine learning in computational histopathology: Challenges and opportunities

Towards More Transparent and Accurate Cancer Diagnosis with an Unsupervised CAE Approach

Contact Info

Product

Resources

About