SMIT: Stochastic Multi-Label Image-to-Image Translation

Abstract

Cross-domain mapping has been a very active topic in recent years. Given one image, its main purpose is to translate it to the desired target domain, or multiple domains in the case of multiple labels. This problem is highly challenging due to three main reasons: (i) unpaired datasets, (ii) multiple attributes, and (iii) the multimodality (e.g. style) associated with the translation. Most of the existing stateof-the-art has focused only on two reasons i.e., either on (i) and (ii), or (i) and (iii). In this work, we propose a joint framework (i, ii, iii) of diversity and multi-mapping image-to-image translations, using a single generator to conditionally produce countless and unique fake images that hold the underlying characteristics of the source image. Our system does not use style regularization, instead, it uses an embedding representation that we call domain embedding for both domain and style. Extensive experiments over different datasets demonstrate the effectiveness of our proposed approach in comparison with the state-ofthe-art in both multi-label and multimodal problems. Additionally, our method is able to generalize under different scenarios: continuous style interpolation, continuous label interpolation, and fine-grained mapping. Code and pretrained models are available at https://github. com/BCV-Uniandes/SMIT.

Night-to-Day Image Translation for Retrieval-based Localization

“…Ignatov et al performed one-way translation for image enhancement, using two discriminators per domain -one for color and the other for texture -rather than just one [10]. And ComboGAN [1] and SMIT [20] allowed for ndomain translation, solving the exponential scaling problem in the number of domains. This project uses ComboGAN as the base image-to-image translation model, which is equivalent to CycleGAN in the case of two domains only.…”

“…Image translation began as altering the characteristics of an image between perceived styles for artistic and/or entertainment purposes. With projects such as [1], [5], [15], [20], [28] breaking ground, it was now possible to perform high-quality image translation. Soon thereafter, the idea was used to aid other learning tasks [9], [15], [18], [19], [21].…”

SESAME: Semantic Editing of Scenes by Adding, Manipulating or Erasing Objects

“…Generative Adversarial Networks [10] have completely revolutionized a great variety of computer vision tasks such as image generation [20,19,30], super resolution [48,27], image attribute manipulation [28,40] and image editing [12,3].…”

Facial expression animation through action units transfer in latent space

“…GAN does a creative work on realistic fake image generation and opens a door to facial attribute editing 30–33 . GAN‐based facial attribute editing approaches have been proposed for unpaired image‐to‐image translation 10–13,34 . The attributes include gender, age, face color, and facial expression 9,35,36 .…”

“…The development of convolutional neural network (CNN) 7 and generative adversarial networks (GAN) 8 has brought facial synthesis great promotion as they can dig deep image feature and generate fake images with high reality. Recently, the technology has been successfully applied to image‐to‐image translation to change the attributes of face, including gender, hair, age, and facial expressions 9–13 . Most of the translation methods take the message‐based approach 14 to describe facial behavior and classify facial expressions into seven basic emotions: anger, disgust, fear, happiness, sadness, surprise, and contempt 15 .…”