Towards Virtual H&E Staining of Hyperspectral Lung Histology Images Using Conditional Generative Adversarial Networks

Abstract

The microscopic image of a specimen in the absence of staining appears colorless and textureless. Therefore, microscopic inspection of tissue requires chemical staining to create contrast. Hematoxylin and eosin (H&E) is the most widely used chemical staining technique in histopathology. However, such staining creates obstacles for automated image analysis systems. Due to different chemical formulations, different scanners, section thickness, and lab protocols, similar tissues can greatly differ in appearance. This huge variability is one of the main challenges in designing robust and resilient automated image analysis systems. Moreover, staining process is time consuming and its chemical effects deform structures of specimens. In this work, we develop a method to virtually stain unstained specimens. Our method utilizes dimension reduction and conditional adversarial generative networks (cGANs) which build highly non-linear mappings between input and output images. Conditional GANs ability to handle very complex functions and high dimensional data enables transforming unstained hyperspectral tissue image to their H&E equivalent which comprises highly diversified appearance. In the long term, such virtual digital H&E staining could automate some of the tasks in the diagnostic pathology workflow which could be used to speed up the sample processing time, reduce costs, prevent adverse effects of chemical stains on tissue specimens, reduce observer variability, and increase objectivity in disease diagnosis.

Distribution Matching Losses Can Hallucinate Features in Medical Image Translation

“…by doctors). Applications include MR to CT [7,8], CS-MRI [9,10], CT to PET [11], and automatic H&E staining [12]. We demonstrate the problem with a caricature example in Figure 1 where we cure cancer (in images) and cause cancer (in images) using a CycleGAN that translates between Flair and T1 MRI samples.…”

Computational Histological Staining and Destaining of Prostate Core Biopsy RGB Images with Generative Adversarial Neural Networks

“…Bayramoglu use the transmittance spectra of hyperspectral nonstained images and corresponding microscopy images of size 1000×1000 px and train a cGAN architecture to report generated H&E-stained lung biopsy images. The generated images are low resolution and suffer from information loss (0.38 SSIM index) [11].…”

“…Bautista et al compare linear and nonlinear mappings of the spectral transmittance data between nonstained and H&E-stained multispectral images resulting in computationally stained multi-spectral images that are then converted to RGB format [10]. Bayramoglu et al use hyperspectral transmittance spectra of the nonstained images and the corresponding microscopy images of H&Estained slides to learn non-linear mappings between these image pairs using a conditional generative adversarial network (cGAN) [11]. Another study describes unsupervised segmentation for low-contrast multichannel color nonstained pathology images by non-linearly mapping such images to an increased number of channels using an empirical kernel map in combination with non-negative matrix factorization [12].…”

Hyperspectral Imaging for the Detection of Glioblastoma Tumor Cells in H&E Slides Using Convolutional Neural Networks

“…Recently, this technology has proven to provide advantages in the diagnosis of different types of diseases [8][9][10]. In the field of histopathological analysis, this technology has been used for different applications, such as the visualization of multiple biological markers within a single tissue specimen with inmunohistochemistry [11][12][13], the digital staining of samples [14,15], or diagnosis.…”