Label-Free Virtual HER2 Immunohistochemical Staining of Breast Tissue using Deep Learning

Abstract

The immunohistochemical (IHC) staining of the human epidermal growth factor receptor 2 (HER2) biomarker is widely practiced in breast tissue analysis, preclinical studies, and diagnostic decisions, guiding cancer treatment and investigation of pathogenesis. HER2 staining demands laborious tissue treatment and chemical processing performed by a histotechnologist, which typically takes one day to prepare in a laboratory, increasing analysis time and associated costs. Here, we describe a deep learning-based virtual HER2 IHC staining method using a conditional generative adversarial network that is trained to rapidly transform autofluorescence microscopic images of unlabeled/label-free breast tissue sections into bright-field equivalent microscopic images, matching the standard HER2 IHC staining that is chemically performed on the same tissue sections. The efficacy of this virtual HER2 staining framework was demonstrated by quantitative analysis, in which three board-certified breast pathologists blindly graded the HER2 scores of virtually stained and immunohistochemically stained HER2 whole slide images (WSIs) to reveal that the HER2 scores determined by inspecting virtual IHC images are as accurate as their immunohistochemically stained counterparts. A second quantitative blinded study performed by the same diagnosticians further revealed that the virtually stained HER2 images exhibit a comparable staining quality in the level of nuclear detail, membrane clearness, and absence of staining artifacts with respect to their immunohistochemically stained counterparts. This virtual HER2 staining framework bypasses the costly, laborious, and time-consuming IHC staining procedures in laboratory and can be extended to other types of biomarkers to accelerate the IHC tissue staining used in life sciences and biomedical workflow.

Virtual Staining of Defocused Autofluorescence Images of Unlabeled Tissue Using Deep Neural Networks

“…By introducing an additional Deep-R network in the inference process, the fast, defocused image virtual staining framework can be implemented on conventional fluorescence microscopes without hardware modifications or a customized optical setup. This fast virtual staining workflow can also be expanded to many other stains, such as Masson's Trichrome stain, Jones' silver stain, and immunohistochemical (IHC) stains [2][3][4]12]. In addition to lung tissue, the presented virtual staining workflow can be applied to other types of human tissue such as, e.g., breast, kidney, salivary gland, liver, and skin [2][3][4].…”

“…This fast virtual staining workflow can also be expanded to many other stains, such as Masson's Trichrome stain, Jones' silver stain, and immunohistochemical (IHC) stains [2][3][4]12]. In addition to lung tissue, the presented virtual staining workflow can be applied to other types of human tissue such as, e.g., breast, kidney, salivary gland, liver, and skin [2][3][4]. Although the virtual staining approach presented here was demonstrated based on the autofluorescence imaging of unlabeled tissue sections, it can also be used to speed up the virtual staining workflow of other label-free microscopy modalities [6,7].…”

“…Additional channels of autofluorescence (e.g., FITC and Cy5 filters in addition to DAPI and TxRed filters) can be used to further improve the inference accuracy of the virtual staining network as demonstrated in Ref. [4] for IHC staining.…”

“…The ability to virtually stain microscopic images of unlabeled tissue sections was demonstrated through deep neural networks, avoiding the laborious and time-consuming histochemical staining processes. These deep learning-based label-free virtual staining methods can use different input imaging modalities, such as autofluorescence microscopy [2][3][4], hyperspectral imaging [5], quantitative phase imaging (QPI) [6], reflectance confocal microscopy [7], and photoacoustic microscopy [8], among others [9][10][11]. Virtual staining, in general, has the potential to be used as a substitute for histochemical staining, providing savings in both costs and tissue processing time.…”

Deep learning-enabled virtual histological staining of biological samples

“…In fact, the autofluorescence emission signatures of biological tissue carry convoluted spatial-spectral information of its metabolic state and pathological condition 29,30 . Therefore, in addition to the standard histochemical stains such as H&E and MT, the autofluorescence images of labelfree tissue can be utilized to generate more complex molecular stains, e.g., highlighting a specific protein expression, as 31 (Fig. 4c), significantly extending the reach of virtual tissue staining via label-free autofluorescence imaging.…”