Self supervised contrastive learning for digital histopathology

Ciga, Ozan; Xu, Tengteng; Martel, Anne L.

doi:10.1016/j.mlwa.2021.100198

Cited by 138 publications

(161 citation statements)

References 35 publications

Supporting

Mentioning

108

Contrasting

Order By: Relevance

“…Generally, different views are augmentations from the same instance. Although several studies [27,22,9] have successfully applied self-supervised contrastive learning to different pathological image analysis tasks, we argue that this paradigm might be suboptimal. The reason mainly comes from two aspects.…”

Section: Introductionmentioning

confidence: 91%

Self-distillation Augmented Masked Autoencoders for Histopathological Image Classification

Luo¹,

Chen²,

Gao³

2022

Preprint

View full text Add to dashboard Cite

Self-supervised learning (SSL) has drawn increasing attention in pathological image analysis in recent years. However, the prevalent contrastive SSL is suboptimal in feature representation under this scenario due to the homogeneous visual appearance. Alternatively, masked autoencoders (MAE) build SSL from a generative paradigm. They are more friendly to pathological image modeling. In this paper, we firstly introduce MAE to pathological image analysis. A novel SD-MAE model is proposed to enable a self-distillation augmented SSL on top of the raw MAE. Besides the reconstruction loss on masked image patches, SD-MAE further imposes the self-distillation loss on visible patches. It guides the encoder to perceive high-level semantics that benefit downstream tasks. We apply SD-MAE to the image classification task on two pathological and one natural image datasets. Experiments demonstrate that SD-MAE performs highly competitive when compared with leading contrastive SSL methods. The results, which are pre-trained using a moderate size of pathological images, are also comparable to the method pre-trained with two orders of magnitude more images. Our code will be released soon.

show abstract

Section: Introductionmentioning

confidence: 91%

Self-distillation Augmented Masked Autoencoders for Histopathological Image Classification

Luo¹,

Chen²,

Gao³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Self-supervised methods have also been applied to histopathology. One approach has been to, without modification, apply methods constructed for natural images (Lu et al (2019); Dehaene et al (2020); Ciga et al (2020)). The generality of these results has, however, been questioned, and for CPC it has been shown to perform sub-optimally for histology images (Stacke et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

Learning Representations with Contrastive Self-Supervised Learning for Histopathology Applications

Stacke¹,

Unger²,

Lundström³

et al. 2021

Preprint

View full text Add to dashboard Cite

Unsupervised learning has made substantial progress over the last few years, especially by means of contrastive self-supervised learning. The dominating dataset for benchmarking self-supervised learning has been ImageNet, for which recent methods are approaching the performance achieved by fully supervised training. The ImageNet dataset is however largely object-centric, and it is not clear yet what potential those methods have on widely different datasets and tasks that are not object-centric, such as in digital pathology. While self-supervised learning has started to be explored within this area with encouraging results, there is reason to look closer at how this setting differs from natural images and ImageNet. In this paper we make an in-depth analysis of contrastive learning for histopathology, pinpointing how the contrastive objective will behave differently due to the characteristics of histopathology data. We bring forward a number of considerations, such as view generation for the contrastive objective and hyper-parameter tuning. In a large battery of experiments, we analyze how the downstream performance in tissue classification will be affected by these considerations. The results point to how contrastive learning can reduce the annotation effort within digital pathology, but that the specific dataset characteristics need to be considered. To take full advantage of the contrastive learning objective, different calibrations of view generation and hyper-parameters are required. Our results pave the way for realizing the full potential of self-supervised learning for histopathology applications. Code and trained models are available at https://github.com/k-stacke/ssl-pathology.

show abstract

“…Self-supervised Learning Recent advances in self-supervised learning (SSL) for computer vision have improved the quality of latent representations in cases without sufficient annotated samples. These methods have shown promising results in medical imaging (Li et al, 2020;Ciga et al, 2022;Dehaene et al, 2020;Sowrirajan et al, 2020;Azizi et al, 2021;. Contrastive SSL is currently the state-of-the-art SSL method for natural images (Chen et al, 2020b,c;Tsai et al, 2021a;Chen et al, 2020a;He et al, 2020;Grill et al, 2020;Caron et al, 2020;Zbontar et al, 2021;Caron et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

“…In this approach, models are trained to increase similarity of representations corresponding to augmented copies of the same image, while decreasing the similarity of augmented copies from different images. Contrastive SSL has been applied to histopathology (Li et al, 2020;Ciga et al, 2022). However, the contrastive loss, InfoNCE (van den Oord et al, 2018) is not tailored to the special properties of WSIs.…”

Section: Related Workmentioning

confidence: 99%

Interpretable Prediction of Lung Squamous Cell Carcinoma Recurrence With Self-supervised Learning

Zhu¹,

Fernandez‐Granda²,

Razavian³

2022

Preprint

View full text Add to dashboard Cite

Lung squamous cell carcinoma (LSCC) has a high recurrence and metastasis rate. Factors influencing recurrence and metastasis are currently unknown and there are no distinct histopathological features indicating the risks of recurrence and metastasis in LSCC. Our study focuses on the recurrence prediction of LSCC based on H&E-stained histopathological whole-slide images (WSI). Due to the small size of LSCC cohorts in terms of patients with available recurrence information, standard end-to-end learning with various convolutional neural networks for this task tends to overfit. Also, the predictions made by these models are hard to interpret. In this work, we propose a novel conditional self-supervised learning (SSL) method to learn representations of WSI at the tile level first, and leverage clustering algorithms to identify the tiles with similar histopathological representations. The resulting representations and clusters from self-supervision are used as features of a survival model for recurrence prediction at the patient level. Using two publicly available datasets from TCGA and CPTAC, we show that our LSCC recurrence prediction survival model outperforms both LSCC pathological stage-based approach and machine learning baselines such as multiple instance learning. The proposed method also enables us to explain the recurrence risk factors via the derived clusters. This can help pathologists derive new hypotheses regarding morphological features associated with LSCC recurrence. Code available at https://github.com/NYUMedML/conditional_ssl_hist.

show abstract

Self supervised contrastive learning for digital histopathology

Cited by 138 publications

References 35 publications

Self-distillation Augmented Masked Autoencoders for Histopathological Image Classification

Self-distillation Augmented Masked Autoencoders for Histopathological Image Classification

Learning Representations with Contrastive Self-Supervised Learning for Histopathology Applications

Interpretable Prediction of Lung Squamous Cell Carcinoma Recurrence With Self-supervised Learning

Contact Info

Product

Resources

About