Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation

Proceedings of the 27th ACM International Conference on Multimedia

Liu

et al. 2019

Self Cite

In this work, we propose a novel Cycle In Cycle Generative Adversarial Network (C 2 GAN) for the task of keypoint-guided image generation. The proposed C 2 GAN is a cross-modal framework exploring a joint exploitation of the keypoint and the image data in an interactive manner. C 2 GAN contains two different types of generators, i.e., keypoint-oriented generator and image-oriented generator. Both of them are mutually connected in an end-to-end learnable fashion and explicitly form three cycled sub-networks, i.e., one image generation cycle and two keypoint generation cycles. Each cycle not only aims at reconstructing the input domain, and also produces useful output involving in the generation of another cycle. By so doing, the cycles constrain each other implicitly, which provides complementary information from the two different modalities and brings extra supervision across cycles, thus facilitating more robust optimization of the whole network. Extensive experimental results on two publicly available datasets, i.e., Radboud Faces [19] and Market-1501 [58], demonstrate that our approach is effective to generate more photo-realistic images compared with state-of-the-art models.

Section: Related Workmentioning

confidence: 99%

Cycle In Cycle Generative Adversarial Networks for Keypoint-Guided Image Generation

Proceedings of the 27th ACM International Conference on Multimedia

Liu

et al. 2019

Self Cite

“…Similar ideas have been proposed in [3,4,5,6]. Despite these efforts, facial expression translation remains a challenging task due to the fact that the expression changes are non-linear [7,8].…”

Section: Introductionmentioning

confidence: 92%

Expression Conditional Gan for Facial Expression-to-Expression Translation

2019 IEEE International Conference on Image Processing (ICIP)

Wang

et al. 2019

Self Cite

In this paper, we focus on the facial expression translation task and propose a novel Expression Conditional GAN (EC-GAN) which can learn the mapping from one image domain to another one based on an additional expression attribute. The proposed ECGAN is a generic framework and is applicable to different expression generation tasks where specific facial expression can be easily controlled by the conditional attribute label. Besides, we introduce a novel face mask loss to reduce the influence of background changing. Moreover, we propose an entire framework for facial expression generation and recognition in the wild, which consists of two modules, i.e., generation and recognition. Finally, we evaluate our framework on several public face datasets in which the subjects have different races, illumination, occlusion, pose, color, content and background conditions. Even though these datasets are very diverse, both the qualitative and quantitative results demonstrate that our approach is able to generate facial expressions accurately and robustly.

“…To further improve the generation performance, the attention mechanism has been recently investigated in image translation, such as [3,45,39,24,26]. However, to the best of our knowledge, our model is the first attempt to incorporate a multi-channel attention selection module within a GAN framework for image-to-image translation task.…”

Section: Related Workmentioning

confidence: 99%

Multi-Channel Attention Selection GAN With Cascaded Semantic Guidance for Cross-View Image Translation

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Sebe

et al. 2019

Self Cite

194

224

Cross-view image translation is challenging because it involves images with drastically different views and severe deformation. In this paper, we propose a novel approach named Multi-Channel Attention SelectionGAN (Selection-GAN) that makes it possible to generate images of natural scenes in arbitrary viewpoints, based on an image of the scene and a novel semantic map. The proposed SelectionGAN explicitly utilizes the semantic information and consists of two stages. In the first stage, the condition image and the target semantic map are fed into a cycled semantic-guided generation network to produce initial coarse results. In the second stage, we refine the initial results by using a multi-channel attention selection mechanism. Moreover, uncertainty maps automatically learned from attentions are used to guide the pixel loss for better network optimization. Extensive experiments on Dayton [42], CVUSA [44] and Ego2Top [1] datasets show that our model is able to generate significantly better results than the state-of-the-art methods. The source code, data and trained models are available at https://github. com/Ha0Tang/SelectionGAN .