Self-Path: Self-Supervision for Classification of Pathology Images With Limited Annotations

Abstract

While high-resolution pathology images lend themselves well to 'data hungry' deep learning algorithms, obtaining exhaustive annotations on these images is a major challenge. In this paper, we propose a self-supervised CNN approach to leverage unlabeled data for learning generalizable and domain invariant representations in pathology images. The proposed approach, which we term as Self-Path, is a multi-task learning approach where the main task is tissue classification and pretext tasks are a variety of self-supervised tasks with labels inherent to the input data. We introduce novel domain specific selfsupervision tasks that leverage contextual, multi-resolution and semantic features in pathology images for semi-supervised learning and domain adaptation. We investigate the effectiveness of Self-Path on 3 different pathology datasets. Our results show that Self-Path with the domain-specific pretext tasks achieves stateof-the-art performance for semi-supervised learning when small amounts of labeled data are available. Further, we show that Self-Path improves domain adaptation for classification of histology image patches when there is no labeled data available for the target domain. This approach can potentially be employed for other applications in computational pathology, where annotation budget is often limited or large amount of unlabeled image data is available.

A digital score of tumour‐associated stroma infiltrating lymphocytes predicts survival in head and neck squamous cell carcinoma

“…The separation between low-and high-risk groups in the C2 and C3 cohorts is more sustained compared with the C1 cohort. The slight difference in performance on C1 may be attributed to stain variation and scanner differences, known as the domain shift problem in the area of computational pathology [35][36][37]. While we have shown the TASIL-score to be prognostically significant across all three cohorts (including C1), the score must be evaluated for robustness to stain variation and scanner differences before it can be deployed in clinical practice.…”

Label Propagation for Annotation-Efficient Nuclei Segmentation from Pathology Images

“…Second, we customize a self-supervised learning method for nuclei segmentation in pathology images. Different from the related methods [36][37][38] that simply applied the SSL techniques from natural images, our method appreciates the special characteristics of the H&E staining method for pathology images, involving the prior knowledge of nuclei. The most relevant work of our nucleiaware colorization method is Yang et al [42], which designed two pretext tasks transforming between the H-component and E-component.…”

“…Recently, we have witnessed the advances of SSL in natural image analysis [31][32][33][34][35], which can be categorized into three types: contrastive learning [31,32], clustering [28,33], and consistency learning [35]. Extensive recent studies [36][37][38] in pathology image analysis have demonstrated the effectiveness of SSL techniques, which were originally proposed for natural images (e.g., CPC [39], SimCLR [31], and MoCo [32]). Subsequently, instead of straightforward extensions, recent studies attempted to develop SSL methods addressing unique problems encountered in pathology images, such as gigapixel whole slide image (WSI) [37] and nuclei counting [40].…”

A 2021 update on cancer image analytics with deep learning

“…Empirical evidence suggests that solving the auxiliary task (e.g., solving a jigsaw) serves as domain‐specific pre‐training by teaching the CNN to extract features that are useful for the main task (e.g., recognizing cancer) as well. Specifically, in cancer image analysis, Self‐Path (Koohbanani, Unnikrishnan, Khurram, Krishnaswamy, & Rajpoot, 2020) used domain specific self‐supervision tasks for effective learning and domain adaptation on histopathology images, while (Sahasrabudhe et al, 2020) used learning to detect patch magnification level as a pretext task for nucleus localization and segmentation.…”

“…While human vision is attuned to glossing over these differences and easily adapt to identify the characteristics that matter—those due to differences in anatomy alone—the accuracy of classification algorithms learned by deep learning often suffers to the point being rendered unusable due to changes in domain. Few‐shot learning and domain adaption techniques (Koohbanani et al, 2020; Larsson et al, 2017; Medela et al, 2019; Noroozi & Favaro, 2016; Pathak et al, 2016; Snell et al, 2017) are therefore of increasing importance if models developed using one cohort are to be successfully applied to another cohort with domain differences.…”