Diversity-Sensitive Conditional Generative Adversarial Networks

Abstract

We propose a simple yet highly effective method that addresses the mode-collapse problem in the Conditional Generative Adversarial Network (cGAN). Although conditional distributions are multi-modal (i.e., having many modes) in practice, most cGAN approaches tend to learn an overly simplified distribution where an input is always mapped to a single output regardless of variations in latent code. To address such issue, we propose to explicitly regularize the generator to produce diverse outputs depending on latent codes. The proposed regularization is simple, general, and can be easily integrated into most conditional GAN objectives. Additionally, explicit regularization on generator allows our method to control a balance between visual quality and diversity. We demonstrate the effectiveness of our method on three conditional generation tasks: image-to-image translation, image inpainting, and future video prediction. We show that simple addition of our regularization to existing models leads to surprisingly diverse generations, substantially outperforming the previous approaches for multi-modal conditional generation specifically designed in each individual task.

Few-shot Image Generation via Cross-domain Correspondence

“…To alleviate mode collapse in GANs, DistanceGAN [2] proposes to preserve the distances between input pairs in the corresponding generated output pairs. A similar scheme has been employed for both unconditional [27,18] and conditional [19,35] generation tasks to increase diversity in the generations. In our work, we aim to inherit the learned diversity from the source model to the target model and achieve this via our novel cross-domain distance consistency loss.…”

CG-NeRF: Conditional Generative Neural Radiance Fields

“…As our method generates images conditioned on extra inputs, variations of the output images are restricted, especially when a color image is given as a condition input. To enable the generator network to produce semantically diverse images based on the condition input, we regularize the generator network with the diversity-sensitive loss (Yang et al 2019a). This is defined as…”

“…Pose Penalty To validate the importance of the posepenalty in relation to the diversity-sensitive loss (Yang et al 2019a) for our method, we conduct an ablation study to confirm the effect of the pose-penalty when attaching the diversity-sensitive loss when training. As shown in Fig.…”

Temporal Prediction and Evaluation of Brassica Growth in the Field using Conditional Generative Adversarial Networks

“…Moreover, we conducted experiments with changed hyperparameter, modifying γ to control the loss weighting (see Fig. 4), or choosing a different architecture such as the Diversity-Sensitive Conditional GAN [55], which is designed to force variability through a different structure and loss functionality. None of these attempts is successful in enforcing more diversity in the generated images.…”