Semi-supervised image attribute editing using generative adversarial networks

Abstract

Image attribute editing is a challenging problem that has been recently studied by many researchers using generative networks. The challenge is in the manipulation of selected attributes of images while preserving the other details. The method to achieve this goal is to find an accurate latent vector representation of an image and a direction corresponding to the attribute. Almost all the works in the literature use labeled datasets in a supervised setting for this purpose. In this study, we introduce an architecture called Cyclic Reverse Generator (CRG), which allows learning the inverse function of the generator accurately via an encoder in an unsupervised setting by utilizing cyclic cost minimization. Attribute editing is then performed using the CRG models for finding desired attribute representations in the latent space. In this work, we use two arbitrary reference images, with and without desired attributes, to compute an attribute direction for editing. We show that the proposed approach performs better in terms of image reconstruction compared to the existing end-to-end generative models both quantitatively and qualitatively. We demonstrate state-of-the-art results on both real images and generated images in MNIST and CelebA datasets.

Towards disease-aware image editing of chest X-rays

“…We then train E, while keeping the weights of G fixed and furthermore using a constant noise realization. During training, the weights of E are updated two times in each iteration as proposed in the original CRG work [2]. In the first update, following the cyclic path depicted with black lines in Fig.…”

“…Generative models such as GANs constitute powerful techniques for capturing the distribution of high-dimensional image data and synthesizing realistic images [8,9,10,15]. GANs have been successfully employed for data augmentation [13] as well as for the manipulation of images in a semantically controlled manner [2,7]. Most studies to date, however, have been restricted to natural images.…”

“…Here, we show that StyleGAN [9] can be trained on the CheXpert [11] dataset to provide a generator G of chest X-rays at 128x128 resolution. Taking inspiration from the recently proposed CRG framework [2], we train a corresponding encoder E and assess the extent to which G and E fulfil the above criteria. In particular, we argue that the medical condition cardiomegaly is at least partially disentangled in latent space, thus permitting disease-aware image manipulation.…”

Surface Defect Detection Method Based on Improved Semisupervised Multitask Generative Adversarial Network

“…To verify the efficacy of the proposed method, comparative evaluation experiments using the NEU-DET dataset and the PCB dataset (self-built dataset) were conducted, and the performance of the proposed iSSMT-GAN is further tested on the NEU-DET dataset and PCB dataset. NEU surface defect is a public defect classification dataset [16], including six different classes from hot-rolled steel plates, such as inclusion, patches, crazing, rolled-in scales, scratches, and pitted surface. Each class has 300 images.…”

“…However, the methods have their own disadvantages. e DNNs, which perform entirely on benchmark datasets, may perform poorly in practical detection [16,17]. Generally, the advantageous performance of these detection methods based on DNN depends on a large amount of training data with excellent labels [18].…”