Asymmetric GAN for Unpaired Image-to-Image Translation

Abstract

Unpaired image-to-image translation problem aims to model the mapping from one domain to another with unpaired training data. Current works like the well-acknowledged Cycle GAN provide a general solution for any two domains through modeling injective mappings with a symmetric structure. While in situations where two domains are asymmetric in complexity, i.e. the amount of information between two domains is different, these approaches pose problems of poor generation quality, mapping ambiguity, and model sensitivity. To address these issues, we propose Asymmetric GAN (AsymGAN) to adapt the asymmetric domains by introducing an auxiliary variable (aux) to learn the extra information for transferring from the information-poor domain to the information-rich domain, which improves the performance of state-of-the-art approaches in the following ways. First, aux better balances the information between two domains which benefits the quality of generation. Second, the imbalance of information commonly leads to mapping ambiguity, where we are able to model one-to-many mappings by tuning aux, and furthermore, our aux is controllable. Third, the training of Cycle GAN can easily make the generator pair sensitive to small disturbances and variations while our model decouples the ill-conditioned relevance of generators by injecting aux during training. We verify the effectiveness of our proposed method both qualitatively and quantitatively on asymmetric situation, labelphoto task, on Cityscapes and Helen datasets, and show many applications of asymmetric image translations. In conclusion, our AsymGAN provides a better solution for unpaired imageto-image translation in asymmetric domains.

Unsupervised many‐to‐many image‐to‐image translation across multiple domains

“…Augmented CycleGAN proposed by Almahairi et al [2], learns stochastic mappings which leverage auxiliary noise to capture multi-modal conditions. Li et al [23] utilize an auxiliary variable to learn the extra information between two domains that have asymmetric information, and then produce diverse target images. Huang et al [14] directly extend UNIT [26] to multimodal scenarios called MUNIT, which encodes the images to a shared content space and combines a random domainspecialized style code for the generation.…”

Deep Generative Adversarial Networks for Image-to-Image Translation: A Review

“…However, DiscoGAN learns to discover relations among different domains using the reconstruction losses, while DualGAN is based on dual learning using the Wasserstein GAN. More recently, AsymGAN [85] has been proposed; it uses an asymmetric framework to model unpaired image-to-image translation between asymmetric domains by adding an auxiliary variable (aux). The aux is used to learn the extra information between the poor and rich domains.…”

Multiautonomous underwater vehicle consistent collaborative hunting method based on generative adversarial network

“…At present, the GAN algorithm is mainly applied to image recognition, such as video recognition 22,23 and image translation. 24,25 Some scholars have applied the GAN model to other fields. Ren and Xu 26 proposed a fully data-driven approach for phasor measurement unit (PMU)-based prefault dynamic security assessment with incomplete data measurements, and it can reduce the impact of data loss on fault assessment.…”