Biopsy-free in vivo virtual histology of skin using deep learning

Li, Jingxi; Garfinkel, Jason; Zhang, Xiaoran; Wu, Di; Zhang, Yijie; Haan, Kevin de; Wang, Hongda; Liu, Tairan; Bai, Bijie; Rivenson, Yair; Rubinstein, Gennady; Scumpia, Philip O.; Özcan, Aydogan

doi:10.1038/s41377-021-00674-8

Cited by 53 publications

(46 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To create pairs of matched data for autofluorescence and bright-field images, we applied a multistage registration algorithm similar to previous works 10,15 . Bright-field images were first downsampled and coarsely matched to autofluorescence images using a correlation-based registration algorithm.…”

Section: Image Acquisition and Preprocessingmentioning

confidence: 99%

Virtual Stain Transfer in Histology via Cascaded Deep Neural Networks

et al. 2022

Self Cite

View full text Add to dashboard Cite

Pathological diagnosis relies on the visual inspection of histologically stained thin tissue specimens, where different types of stains are applied to bring contrast to and highlight various desired histological features. However, the destructive histochemical staining procedures are usually irreversible, making it very difficult to obtain multiple stains on the same tissue section. Here, we demonstrate a virtual stain transfer framework via a cascaded deep neural network (C-DNN) to digitally transform hematoxylin and eosin (H&E) stained tissue images into other types of histological stains. Unlike a single neural network structure which only takes one stain type as input to digitally output images of another stain type, C-DNN first uses virtual staining to transform autofluorescence microscopy images into H&E and then performs stain transfer from H&E to the domain of the other stain in a cascaded manner. This cascaded structure in the training phase allows the model to directly exploit histochemically stained image data on both H&E and the target special stain of interest. This advantage alleviates the challenge of paired data acquisition and improves the image quality and color accuracy of the virtual stain transfer from H&E to another stain. We validated the superior performance of this C-DNN approach using kidney needle core biopsy tissue sections and successfully transferred the H&E-stained tissue images into virtual PAS (periodic acid-Schiff) stain. This method provides high-quality virtual images of special stains using existing, histochemically stained slides and creates new opportunities in digital pathology by performing highly accurate stain-to-stain transformations.

show abstract

Section: Image Acquisition and Preprocessingmentioning

confidence: 99%

Virtual Stain Transfer in Histology via Cascaded Deep Neural Networks

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Clinically, hematoxylin and eosin (H&E) stain is the most frequently used routine stain which can provide rich information for visualizing nuclear and cytoplasmic components, and are widely used to diagnose cancer. However, conventional sample preparation involves many time-consuming and laborious Therefore, many researchers in the field of biomedical engineering are spending their effort on generating the standard H&E-stained images from the images acquired by a label-free imaging method (e.g., autofluorescence imaging) through virtual staining enabled by deep learning [39], [40], [41], [42].…”

Section: Image Style Transformation With Training Images Requiring Re...mentioning

confidence: 99%

A Weakly Supervised Deep Generative Model for Complex Image Restoration and Style Transformation

Dai¹,

Wong²,

Wong³

2022

Preprint

View full text Add to dashboard Cite

<p>The datasets for transforming autofluorescence images to the histochemically stained images were acquired from a human breast biopsy tissue and a human liver cancer tissue. Tissues of breast cancer and liver cancer were extracted surgically or through tissue biopsy. The tissues were formalin-fixed and paraffin-embedded (FFPE). Thin tissue slices, with a thickness of 4 µm, were sectioned and placed on a quartz slide. The tissue slices were deparaffined prior to imaging. The autofluorescence images were acquired from a wide-field inverted microscope equipped with a 10X/0.3 numerical aperture (NA) objective lens (Plan Fluorite, Olympus Corp.), an infinity-corrected tube lens (TTL-180-A, Thorlabs Inc.), and a monochrome scientific complementary metal-oxide-semiconductor camera (pco.panda 4.2, PCO. Inc.). A deep ultraviolet light-emitting diode of 265 nm (M265L4, Thorlabs Inc.) was used as an excitation light source because of its high absorption in cell nuclei [28], consequently providing high nuclear contrast without labels [29]. After acquiring the autofluorescence image, the same slide was stained with H&E and its bright-field images were captured using a whole-slide scanner equipped with a 20X/0.75 NA objective lens (NanoZoomer-SQ, Hamamatsu Photonics K.K). All human experiments were carried out in conformity with a clinical research ethics review approved by the Institutional Review Board of the Chinese University of Hong Kong/ New Territories East Cluster (reference number: 2021.597).</p>

show abstract

“…Several optical microscopes have been developed to provide H&E-like imaging capabilities, such as quantitative phase imaging 6 , reflectance confocal microscopy (RCM) 7 , microscopy with ultraviolet surface excitation (MUSE) 8 , 9 , optical coherence tomography (OCT) 10 – 12 , autofluorescence microscopy 13 , and photoacoustic microscopy (PAM) 14 – 17 . These methods are usually coupled with deep learning-based virtual staining models, particularly generative adversarial networks (GANs) 18 .…”

Section: Introductionmentioning

confidence: 99%

Virtual histological staining of label-free total absorption photoacoustic remote sensing (TA-PARS)

Boktor

Ecclestone

Pekar

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Histopathological visualizations are a pillar of modern medicine and biological research. Surgical oncology relies exclusively on post-operative histology to determine definitive surgical success and guide adjuvant treatments. The current histology workflow is based on bright-field microscopic assessment of histochemical stained tissues and has some major limitations. For example, the preparation of stained specimens for brightfield assessment requires lengthy sample processing, delaying interventions for days or even weeks. Therefore, there is a pressing need for improved histopathology methods. In this paper, we present a deep-learning-based approach for virtual label-free histochemical staining of total-absorption photoacoustic remote sensing (TA-PARS) images of unstained tissue. TA-PARS provides an array of directly measured label-free contrasts such as scattering and total absorption (radiative and non-radiative), ideal for developing H&E colorizations without the need to infer arbitrary tissue structures. We use a Pix2Pix generative adversarial network to develop visualizations analogous to H&E staining from label-free TA-PARS images. Thin sections of human skin tissue were first virtually stained with the TA-PARS, then were chemically stained with H&E producing a one-to-one comparison between the virtual and chemical staining. The one-to-one matched virtually- and chemically- stained images exhibit high concordance validating the digital colorization of the TA-PARS images against the gold standard H&E. TA-PARS images were reviewed by four dermatologic pathologists who confirmed they are of diagnostic quality, and that resolution, contrast, and color permitted interpretation as if they were H&E. The presented approach paves the way for the development of TA-PARS slide-free histological imaging, which promises to dramatically reduce the time from specimen resection to histological imaging.

show abstract

Biopsy-free in vivo virtual histology of skin using deep learning

Cited by 53 publications

References 61 publications

Virtual Stain Transfer in Histology via Cascaded Deep Neural Networks

Virtual Stain Transfer in Histology via Cascaded Deep Neural Networks

A Weakly Supervised Deep Generative Model for Complex Image Restoration and Style Transformation

Virtual histological staining of label-free total absorption photoacoustic remote sensing (TA-PARS)

Contact Info

Product

Resources

About