Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning

Rivenson, Yair; Wang, Hongda; Wei, Zhensong; Haan, Kevin de; Zhang, Hongjie; Wu, Yichen; Günaydın, Harun; Zuckerman, Jonathan E.; Chong, Thomas; Sisk, Anthony; Westbrook, Lindsey; Wallace, W. Dean; Özcan, Aydogan

doi:10.1038/s41551-019-0362-y

Cited by 431 publications

(364 citation statements)

References 32 publications

Supporting

Mentioning

362

Contrasting

Order By: Relevance

“…S1). Previous studies 28,29,42 have demonstrated that a virtual spatialization map (VSM), covering the entire WSI can be defined on the basis of CNN models. These heatmaps reflect the importance score assigned to each tile used in the algorithm.…”

Section: A Deep Learning Model For the Prediction Of Gene Expressionmentioning

confidence: 99%

“…We then investigated how HE2RNA learned to recognize important histological patterns within the slides, by using our model to generate heatmaps corresponding to the most predictive tiles used to predict a gene's expression. Such approaches could also be used to detect histological subtypes 10 , genetic mutations 28 or to map the infiltration of tumors by tumor-infiltrating lymphocytes (TILs) 29 or other immune cells (e.g. macrophages, NK cells) based on cell-specific gene signatures .…”

mentioning

confidence: 99%

“…This model also sets the stage for possible future applications such as virtual staining in predicting level of expression of a specific molecular marker from the WSI. Such approaches could also be used to detect histological subtypes 10 , genetic mutations 28 or to map the infiltration of tumors by tumor-infiltrating lymphocytes (TILs) 29 or other immune cells (e.g. macrophages, NK cells) based on cell-specific gene signatures .…”

mentioning

confidence: 99%

See 2 more Smart Citations

Transcriptomic learning for digital pathology

Schmauch¹,

Romagnoni²,

Pronier³

et al. 2019

Preprint

View full text Add to dashboard Cite

^,⋕ : These authors contributed equally.Deep learning methods for digital pathology analysis have proved an effective way to address multiple clinical questions, from diagnosis to prognosis and the prediction of treatment outcomes. They have also recently been used to predict gene mutations from pathology images, but no comprehensive evaluation of their potential for extracting molecular features from histology slides, has yet been performed. We propose a novel approach based on the integration of multiple data modes, and show that our deep learning model HE2RNA can be trained to predict systematically RNA-Seq profiles from whole-slide images alone, without the need for expert annotation. The model facilitates the virtual spatialization of gene expression, as validated by double-staining in an independent dataset. The results can therefore be interpreted in detail and this model opens up new opportunities for virtual staining. Finally, the transcriptomic representation learned by the model could be could be used to improve performances for other clinical tasks, particularly for small datasets. For example we studied the problem of predicting microsatellite instability from Hematoxylin & Eosin (H&E)-stained images. Greater prediction ability was achieved in such a realistic framework.

show abstract

Section: A Deep Learning Model For the Prediction Of Gene Expressionmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Transcriptomic learning for digital pathology

Schmauch¹,

Romagnoni²,

Pronier³

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…This open-source package gives biologists access to a wide variety of user-friendly image analysis tools through third-party plugins and macros.Recent works have aimed at providing a link between TensorFlow 1 and ImageJ 2 . In particular, the CSBDeep team [5], the ImageJ2 team [6], and the Ozcan Research Group [7,8] have pioneered this connection by making their pre-trained TensorFlow models accessible through ImageJ. Unfortunately, this connection effort has remained restricted to their specific applications.We present DeepImageJ 3 , an open-source plugin of ImageJ that runs a variety of third-party TensorFlow models in a generic way.…”

mentioning

confidence: 99%

“…Recent works have aimed at providing a link between TensorFlow 1 and ImageJ 2 . In particular, the CSBDeep team [5], the ImageJ2 team [6], and the Ozcan Research Group [7,8] have pioneered this connection by making their pre-trained TensorFlow models accessible through ImageJ. Unfortunately, this connection effort has remained restricted to their specific applications.…”

mentioning

confidence: 99%

DeepImageJ: A user-friendly environment to run deep learning models in ImageJ

Mariscal

Garcia-Lopez-de-Haro²,

Donati³

et al. 2019

Preprint

View full text Add to dashboard Cite

DeepImageJ is a user-friendly plugin that enables the generic use in FIJI/ImageJ of pretrained deep learning (DL) models provided by their developers. The plugin acts as a software layer between TensorFlow and FIJI/ImageJ, runs on a standard CPU-based computer and can be used without any DL expertise. Beyond its direct use, we expect DeepImageJ to contribute to the spread and assessment of DL models in life-sciences applications and bioimage informatics.Deep learning (DL) models have a profound impact on a wide range of imaging applications, including life-sciences [1, 2, 3]. Unfortunately, their deployment is often riddled with technical challenges for the non-expert user. Their appropriate use requires insights on sophisticated DL concepts and good programming skills.This situation is in sharp contrast with the philosophy behind ImageJ, the de-facto standard image processing software in life-sciences [4]. This open-source package gives biologists access to a wide variety of user-friendly image analysis tools through third-party plugins and macros.Recent works have aimed at providing a link between TensorFlow 1 and ImageJ 2 . In particular, the CSBDeep team [5], the ImageJ2 team [6], and the Ozcan Research Group [7,8] have pioneered this connection by making their pre-trained TensorFlow models accessible through ImageJ. Unfortunately, this connection effort has remained restricted to their specific applications.We present DeepImageJ 3 , an open-source plugin of ImageJ that runs a variety of third-party TensorFlow models in a generic way. Keras models can also be integrated whenever properly converted to TensorFlow's format SavedModels. DeepImageJ is designed as a standard ImageJ plugin with the technicalities hidden behind a user-friendly interface.

show abstract

Use of Deep Learning to Develop and Analyze Computational Hematoxylin and Eosin Staining of Prostate Core Biopsy Images for Tumor Diagnosis

et al. 2020

View full text Add to dashboard Cite

IMPORTANCE Histopathological diagnoses of tumors from tissue biopsy after hematoxylin and eosin (H&E) dye staining is the criterion standard for oncological care, but H&E staining requires trained operators, dyes and reagents, and precious tissue samples that cannot be reused. OBJECTIVES To use deep learning algorithms to develop models that perform accurate computational H&E staining of native nonstained prostate core biopsy images and to develop methods for interpretation of H&E staining deep learning models and analysis of computationally stained images by computer vision and clinical approaches. DESIGN, SETTING, AND PARTICIPANTS This cross-sectional study used hundreds of thousands of native nonstained RGB (red, green, and blue channel) whole slide image (WSI) patches of prostate core tissue biopsies obtained from excess tissue material from prostate core biopsies performed in

show abstract

Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning

Cited by 431 publications

References 32 publications

Transcriptomic learning for digital pathology

Transcriptomic learning for digital pathology

DeepImageJ: A user-friendly environment to run deep learning models in ImageJ

Use of Deep Learning to Develop and Analyze Computational Hematoxylin and Eosin Staining of Prostate Core Biopsy Images for Tumor Diagnosis

Contact Info

Product

Resources

About