Transcriptomic learning for digital pathology

Abstract

^,⋕ : These authors contributed equally.Deep learning methods for digital pathology analysis have proved an effective way to address multiple clinical questions, from diagnosis to prognosis and the prediction of treatment outcomes. They have also recently been used to predict gene mutations from pathology images, but no comprehensive evaluation of their potential for extracting molecular features from histology slides, has yet been performed. We propose a novel approach based on the integration of multiple data modes, and show that our deep learning model HE2RNA can be trained to predict systematically RNA-Seq profiles from whole-slide images alone, without the need for expert annotation. The model facilitates the virtual spatialization of gene expression, as validated by double-staining in an independent dataset. The results can therefore be interpreted in detail and this model opens up new opportunities for virtual staining. Finally, the transcriptomic representation learned by the model could be could be used to improve performances for other clinical tasks, particularly for small datasets. For example we studied the problem of predicting microsatellite instability from Hematoxylin & Eosin (H&E)-stained images. Greater prediction ability was achieved in such a realistic framework.

Clinical-Grade Detection of Microsatellite Instability in Colorectal Tumors by Deep Learning

“…15 In contrast, computerbased image analysis by deep learning has enabled robust detection of MSI and dMMR status directly from routine H&E histology: we recently presented 16 and later refined 17 such a deep learning assay, which was independently validated by 2 other groups. 18,19 However, all of these studies used a few hundred patients with CRC at most, but clinical implementation of a deep learning-based diagnostic assay requires enhanced sensitivity and specificity to those previously reported and large-scale validation across multiple populations in different countries.…”

“…Smaller proof-of-concept studies have shown that deep learning can detect a range of molecular biomarkers directly from routine histology, including multiple clinically relevant oncogenes. [17][18][19] However, these classifiers were not validated in large multicenter cohorts and cannot be readily generalized beyond the training set. To our knowledge, the present study is the first international collaborative effort to validate such a deep learning-based molecular biomarker.…”

Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis

“…However, once learned, the feature representation may also be used to find similar images 11 and to quantify associations with traits beyond tissue types 12,13 . This approach, known as transfer learning, has been used to establish associations with genomic alterations [14][15][16][17][18][19] , transcriptomic changes 20,21 and survival [22][23][24] .…”

Imputing missing RNA-sequencing data from DNA methylation by using a transfer learning–based neural network

“…In previous study of genome-wide association analysis without directly measured gene expression [ 6 , 33 ], gene expressions could be imputed from genomic data to perform transcriptome-wide association analysis that can reduce the multiple-testing burden and identify associated genes. Recently, many studies have been proposed to impute gene expressions from genomic data, or even from pathology images [ 34 ]. In the future, the predicted gene expression from other omics data, such as genomics, pathology, or/and radiomics, can also be integrated in epigenome-wide association studies [ 35 ].…”