Region-Guided CycleGANs for Stain Transfer in Whole Slide Images

Abstract

In whole slide imaging, commonly used staining techniques based on hematoxylin and eosin (H&E) and immunohistochemistry (IHC) stains accentuate different aspects of the tissue landscape. In the case of detecting metastases, IHC provides a distinct readout that is readily interpretable by pathologists. IHC, however, is a more expensive approach and not available at all medical centers. Virtually generating IHC images from H&E using deep neural networks thus becomes an attractive alternative. Deep generative models such as CycleGANs learn a semantically-consistent mapping between two image domains, while emulating the textural properties of each domain. They are therefore a suitable choice for stain transfer applications. However, they remain fully unsupervised, and possess no mechanism for enforcing biological consistency in stain transfer. In this paper, we propose an extension to CycleGANs in the form of a region of interest discriminator. This allows the CycleGAN to learn from unpaired datasets where, in addition, there is a partial annotation of objects for which one wishes to enforce consistency. We present a use case on whole slide images, where an IHC stain provides an experimentally generated signal for metastatic cells. We demonstrate the superiority of our approach over prior art in stain transfer on histopathology tiles over two datasets. Our code and model are available at https://github.com/jcboyd/miccai2022-roigan.

Multiplexed tumor profiling with generative AI accelerates histopathology workflows and improves clinical predictions

“…Local consistency: The local consistency objective aims to enforce biological consistency at a local, cell-level and consists of two loss terms, namely a cell discriminator loss (L cellDisc ) and a cell classification loss (L cellClass ) (Figure 2B, panel 3). The cell discriminator loss is inspired by [26], and uses the cell discriminator D cell to identify whether a cell is real or virtual, in the same way that the patch discriminator of Eq. ( 1) attempts to classify patches as real or virtual.…”

“…Additionally, as tissue architecture largely alters after the first set of slices, retrospective addition of new markers or multiplexing of several markers on a specific area/focal plane of interest is impossible. To circumvent these limitations, unpaired stain-to-stain (S2S) translation models have recently emerged, with early applications in translating from H&E to IHC [22][23][24][25][26] and special staining [27,28] and from cryosections to Formalin-Fixed Paraffin-Embedded (FFPE) sections [29]. The vast majority of unpaired S2S translation models are inspired by CycleGAN [30]; they depend on an adversarial loss to preserve the source content (tissue architecture), and a cycle consistency loss to preserve the target style (staining pattern).…”

“…To account for staining unreliability, recent works exploit expert annotations in the source and target domains to guide the translation. For example, Boyd et al [26] translated H&E to Cytokeratin (CK) stained IHC using a region-based CycleGAN and expert annotations of positive and negative metastatic regions on the H&E images. Similarly, Liu et al [25] translated H&E to Ki67 stained IHC by leveraging cancer or normal region annotations in both the H&E and IHC images.…”

Histopathological Domain Adaptation with Generative Adversarial Networks Bridging the Domain Gap Between Thyroid Cancer Histopathology Datasets

“…Generative adversarial networks (GANs) were introduced by Goodfellow et al (32) as a method of producing high quality synthetic (fake) images which approximate an underlying real data distribution. Adversarial training methods have been applied successfully in the field of medical imaging in areas such as MRI reconstruction and tumour segmentation (33), X-ray organ segmentation (34,35), and virtual slide staining (36)(37)(38). GAN-generated synthetic images have been proven to pass visual Turing tests when presented to trained medical professionals.…”