Improving Unsupervised Stain-To-Stain Translation using Self-Supervision and Meta-Learning

Bouteldja, Nassim; Klinkhammer, Barbara M.; Schlaich, Tarek; Merhof, Dorit

doi:10.48550/arxiv.2112.08837

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The SSIM-guided cGAN algorithm we have developed is capable of recapitulating novel protein markers in large multiplexed histologies that were not experimentally included in the IHC. This method of stain-to-stain translation has been suggested in a previous study Bao et al [2021], Bouteldja et al [2021], Salehi and Chalechale [2020], Ghahremani et al [2021] but we demonstrate that using CODEX/Phenocyler data of up to 29 multiplexed protein channels results in medically relevant loss and photo-accurate images. We confirm the ability for our cGAN to generate accurate protein stains through low pixel variance in predicted DAPI channels of HuBMAP and cancer data samples.…”

Section: Discussionmentioning

confidence: 48%

A SSIM Guided cGAN Architecture For Clinically Driven Generative Image Synthesis of Multiplexed Spatial Proteomics Channels

Saurav¹,

Nasr²,

Paul³

et al. 2022

Preprint

View full text Add to dashboard Cite

Histopathological work in clinical labs relying on immunostaining of proteins represents a bottleneck in processing medical tissue samples. Multiplexed spatial proteomics imaging can increase interpretive power but cannot cost-effectively sample the entire proteomic retinue important to diagnostic medicine or drug development. Here we present a structural similarity index measure (SSIM) guided conditional Generative Adversarial Network (cGAN) that generatively performs image-to-image (i2i) synthesis to generate photo-accurate protein channels in multiplexed spatial proteomics images. This approach can be utilized to accurately generate missing spatial proteomics channels that were not included during experimental data collection either at the bench or the clinic. Experimental spatial proteomic data from the Human BioMolecular Atlas Program (HuBMAP) was used to generate spatial representations of missing proteins through a U-Net based image synthesis pipeline. HuBMAP channels were hierarchically clustered by the (SSIM) as a heuristic to obtain the minimal set needed to recapitulate the underlying biology represented by the spatial landscape of proteins. We subsequently prove that our SSIM based architecture allows for scaling of generative image synthesis to slides with up to 100 channels, which is better than current state of the art algorithms which are limited to data with 11 channels. We validate these claims by generating a new experimental spatial proteomics data set from human lung adenocarcinoma tissue sections and show that a model trained on HuBMAP can accurately synthesize channels from our new data set. The ability to recapitulate experimental data from sparsely stained multiplexed histological slides containing spatial proteomic will have tremendous impact on medical diagnostics and drug development, and also raises important questions on the medical ethics of utilizing data produced by generative image synthesis in the clinical setting. The algorithm that we present in this paper will allow researchers and clinicians to save time and Preprint. Under review.

show abstract

Section: Discussionmentioning

confidence: 48%