Unsupervised data to content transformation with histogram-matching cycle-consistent generative adversarial networks

Abstract

The segmentation of images is a common task in a broad range of research fields. To tackle increasingly complex images, artificial intelligence (AI) based approaches have emerged to overcome the shortcomings of traditional feature detection methods. Owing to the fact that most AI research is made publicly accessible and programming the required algorithms is now possible in many popular languages, the use of such approaches is becoming widespread. However, these methods often require data labeled by the researcher to provide a training target for the algorithms to converge to the desired result. This labeling is a limiting factor in many cases and can become prohibitively time consuming. Inspired by Cycle-consistent Generative Adversarial Networks' (cycleGAN) ability to perform style transfer, we outline a method whereby a computer generated set of images is used to segment the true images. We benchmark our unsupervised approach against a state-of-the-art supervised cell-counting network on the VGG Cells dataset and show that it is not only competitive but can also precisely locate individual cells. We demonstrate the power of this method by segmenting bright-field images of cell cultures, a live-dead assay of C. elegans and X-ray-computed tomography of metallic nanowire meshes.

Non-disruptive collagen characterization in clinical histopathology using cross-modality image synthesis

“…Xu et al proposed a novel GAN-based approach to convert the H&E staining of WSIs to virtual immunohistochemistry staining based on cytokeratins 18 and 19, an approach that potentially obviates the need for destructive immunohistochemistry-based tissue testing 38 . Ilhe et al used Cycle-GAN for segmenting unlabeled data of VGG cells, VGG Cells dataset, bright-field images of cell cultures, a live-dead assay of C. Elegans and X-ray-computed tomography of metallic nanowire meshes 39 .…”

The role of convolutional neural networks in scanning probe microscopy: a review

“…With training, the discriminator improves, and the generator learns to generate fake images that look real. This process repeats itself until the discriminator cannot distinguish between the fake images generated and the real ones [ 103 – 104 ]. GANs have been used to achieve segmentation without human annotation [ 104 ].…”

“…This process repeats itself until the discriminator cannot distinguish between the fake images generated and the real ones [ 103 – 104 ]. GANs have been used to achieve segmentation without human annotation [ 104 ]. In this approach, computer-generated images are used for segmenting true bright-field images by synthesizing colorful segmented images and then overlaying them on the experimental image.…”

ASIST: Annotation-free Synthetic Instance Segmentation and Tracking by Adversarial Simulations

“…To tackle the limitations of needing paired training data requirements, CycleGAN [20] was proposed to further advance the GAN technique to broader applications. Cycle-GAN has shown promise in on cross-modality synthesis [21] and microscope image synthesis [22]. DeepSynth [23] demonstrated that CycleGAN can be applied to 3D medical image synthesis.…”