Self-supervised attention-based deep learning for pan-cancer mutation prediction from histopathology

Saldanha, Oliver Lester; Loeffler, Chiara Maria Lavinia; Niehues, Jan Moritz; Treeck, Marko van; Seraphin, Tobias Paul; Hewitt, Kim; Cifci, Didem; Veldhuizen, Gregory Patrick; Ramesh, Siddhi; Pearson, Alexander T.; Kather, Jakob Nikolas

doi:10.1038/s41698-023-00365-0

Cited by 26 publications

(15 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sub-images of live tumors that are free of artifacts and contain descriptive histological features are extracted, which are further used to train and test the deep neural network. The output layers of the network were configured in 2 different ways: a final Cox model layer for [13] TCGA CPTAC TCGA (n = 397) CPTAC (n = 96) AUC (0.84 ± 0.06) SSL + attMIL Yes Rathore et al (2019) [14] TCGA TCGA (n = 663) AUC (0.86) ResNet No…”

Section: Prediction Of Idh For Glioma Wsi Based On Resnet Network Arc...mentioning

confidence: 99%

Insight into deep learning for glioma IDH medical image analysis: A systematic review

Lv,

Liu,

Sun

et al. 2024

Medicine

View full text Add to dashboard Cite

Background: Deep learning techniques explain the enormous potential of medical image analysis, particularly in digital pathology. Concurrently, molecular markers have gained increasing significance over the past decade in the context of glioma patients, providing novel insights into diagnosis and more personalized treatment options. Deep learning combined with imaging and molecular analysis enables more accurate prognostication of patients, more accurate treatment plan proposals, and accurate biomarker (IDH) prediction for gliomas. This systematic study examines the development of deep learning techniques for IDH prediction using histopathology images, spanning the period from 2019 to 2023. Method: The study adhered to the PRISMA reporting requirements, and databases including PubMed, Google Scholar, Google Search, and preprint repositories (such as arXiv) were systematically queried for pertinent literature spanning the period from 2019 to the 30th of 2023. Search phrases related to deep learning, digital pathology, glioma, and IDH were collaboratively utilized. Results: Fifteen papers meeting the inclusion criteria were included in the analysis. These criteria specifically encompassed studies utilizing deep learning for the analysis of hematoxylin and eosin images to determine the IDH status in patients with gliomas. Conclusions: When predicting the status of IDH, the classifier built on digital pathological images demonstrates exceptional performance. The study’s predictive effectiveness is enhanced with the utilization of the appropriate deep learning model. However, external verification is necessary to showcase their resilience and universality. Larger sample sizes and multicenter samples are necessary for more comprehensive research to evaluate performance and confirm clinical advantages.

show abstract

Section: Prediction Of Idh For Glioma Wsi Based On Resnet Network Arc...mentioning

confidence: 99%

Insight into deep learning for glioma IDH medical image analysis: A systematic review

Lv,

Liu,

Sun

et al. 2024

Medicine

View full text Add to dashboard Cite

show abstract

“…MSI and HRD are both abnormalities impacting the deoxyribonucleic acid damage repair (DDR) process in tumors. Early recognition of those biomarkers may benefit the patients through specific therapies targeting DDR-related genomic alterations (19, 20). This is of particular interest in breast cancer (21, 56) and colorectal cancer (11, 21).…”

Section: Experimental and Evaluation Setupmentioning

confidence: 99%

Scaling Self-Supervised Learning for Histopathology with Masked Image Modeling

Filiot¹,

Ghermi²,

Olivier³

et al. 2023

Preprint

View full text Add to dashboard Cite

Computational pathology is revolutionizing the field of pathology by integrating advanced computer vision and machine learning technologies into diagnostic workflows. Recently, self-supervised learning (SSL) has emerged as a promising solution to learn representations from histology patches, leveraging large volumes of unannotated whole slide images (WSI). In particular, Masked Image Modeling (MIM) showed remarkable results and robustness over purely contrastive learning methods. In this work, we explore the application of MIM to histology using iBOT, a self-supervised transformer-based framework. Through a wide range of downstream tasks over seven cancer indications, we provide recommendations on the pre-training of large models for histology data using MIM. First, we demonstrate that in-domain pre-training with iBOT outperforms both ImageNet pre-training and a model pre-trained with a purely contrastive learning objective, MoCo V2. Second, we show that Vision Transformers models (ViT), when scaled appropriately, have the capability to learn pan-cancer representations that benefit a large variety of downstream tasks. Finally, our iBOT ViT-Base model, pre-trained on more than 40 million histology images from 16 different cancer types, achieves state-of-the-art performance in most weakly-supervised WSI classification tasks compared to other SSL frameworks.

show abstract

“…This initial step establishes a foundation model that can be adapted to various downstream tasks using limited numbers of training samples. As such, this approach is particularly useful in the field of histopathology and is increasingly being adopted, given the limited availability of labeled samples [34,35].…”

Section: Model Trainingmentioning

confidence: 99%

Image-Based Deep Learning Detection of High-Grade B-Cell Lymphomas Directly from Hematoxylin and Eosin Images

Perry,

Greenberg,

Haberman

et al. 2023

Cancers

View full text Add to dashboard Cite

Deep learning applications are emerging as promising new tools that can support the diagnosis and classification of different cancer types. While such solutions hold great potential for hematological malignancies, there have been limited studies describing the use of such applications in this field. The rapid diagnosis of double/triple-hit lymphomas (DHLs/THLs) involving MYC, BCL2 and/or BCL6 rearrangements is obligatory for optimal patient care. Here, we present a novel deep learning tool for diagnosing DHLs/THLs directly from scanned images of biopsy slides. A total of 57 biopsies, including 32 in a training set (including five DH lymphoma cases) and 25 in a validation set (including 10 DH/TH cases), were included. The DHL-classifier demonstrated a sensitivity of 100%, a specificity of 87% and an AUC of 0.95, with only two false positive cases, compared to FISH. The DHL-classifier showed a 92% predictive value as a screening tool for performing conventional FISH analysis, over-performing currently used criteria. The work presented here provides the proof of concept for the potential use of an AI tool for the identification of DH/TH events. However, more extensive follow-up studies are required to assess the robustness of this tool and achieve high performances in a diverse population.

show abstract

Self-supervised attention-based deep learning for pan-cancer mutation prediction from histopathology

Cited by 26 publications

References 39 publications

Insight into deep learning for glioma IDH medical image analysis: A systematic review

Insight into deep learning for glioma IDH medical image analysis: A systematic review

Scaling Self-Supervised Learning for Histopathology with Masked Image Modeling

Image-Based Deep Learning Detection of High-Grade B-Cell Lymphomas Directly from Hematoxylin and Eosin Images

Contact Info

Product

Resources

About