Deformable GANs for Pose-Based Human Image Generation

Siarohin, Aliaksandr; Sangineto, Enver; Lathuilière, Stéphane; Sebe, Nicu

doi:10.1109/cvpr.2018.00359

Cited by 498 publications

(681 citation statements)

References 23 publications

Supporting

Mentioning

659

Contrasting

Order By: Relevance

“…We can see that the proposed AsymmetricGAN produces much more photo-realistic results with convincing details compared with other approaches, i.e., GestureGAN [10], PG 2 [8], DPIG [26], PoseGAN [9] and SAMG [50]. Moreover, we provide quantitative comparison with those methods.…”

Section: B Hand Gesture-to-gesture Translation Taskmentioning

confidence: 78%

See 1 more Smart Citation

Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation

Tang

Sebe

et al. 2019

2019 International Joint Conference on Neural Networks (IJCNN)

Self Cite

126

View full text Add to dashboard Cite

State-of-the-art models for unpaired image-to-image translation with Generative Adversarial Networks (GANs) can learn the mapping from the source domain to the target domain using a cycle-consistency loss. The intuition behind these models is that if we translate from one domain to the other and back again we should arrive at where we started. However, existing methods always adopt a symmetric network architecture to learn both forward and backward cycles. Because of the task complexity and cycle input difference between the source and target image domains, the inequality in bidirectional forward-backward cycle translations is significant and the amount of information between two domains is different. In this paper, we analyze the limitation of the existing symmetric GAN models in asymmetric translation tasks, and propose an AsymmetricGAN model with both translation and reconstruction generators of unequal sizes and different parameter-sharing strategy to adapt to the asymmetric need in both unsupervised and supervised image-to-image translation tasks. Moreover, the training stage of existing methods has the common problem of model collapse that degrades the quality of the generated images, thus we explore different optimization losses for better training of AsymmetricGAN, and thus make image-to-image translation with higher consistency and better stability. Extensive experiments on both supervised and unsupervised generative tasks with several publicly available datasets demonstrate that the proposed AsymmetricGAN achieves superior model capacity and better generation performance compared with existing GAN models. To the best of our knowledge, we are the first to investigate the asymmetric GAN framework on both unsupervised and supervised image-to-image translation tasks. The source code, data and trained models are available at https://github.com/Ha0Tang/AsymmetricGAN.

show abstract

Section: B Hand Gesture-to-gesture Translation Taskmentioning

confidence: 78%

“…Similar ideas have also been applied to many other tasks, e.g., pose-guided person image generation [8], [26], [9] and hand gesture-to-gesture translation [10]. However, all of these models require paired training data, which are usually costly to obtain.…”

Section: Related Workmentioning

confidence: 99%

Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation

Tang

Sebe

et al. 2019

2019 International Joint Conference on Neural Networks (IJCNN)

Self Cite

126

View full text Add to dashboard Cite

show abstract

“…Human image synthesis, including human motion imitation [1,19,31], appearance transfer [26,37] and novel * Contributed equally and work done while Wen Liu was a Research Intern with Tencent AI Lab. Figure 1.…”

Section: Introductionmentioning

confidence: 99%

“…Texture warping, however, could not preserve the source information as well, in terms of the color, style or face identity, because the generator might drop out source information after several down-sampling operations, such as stride convolution and pooling. Meanwhile, contemporary works [4,31] propose to warp the deep features of the source images into target pose rather than that in image space, as shown in Fig 2 (c), named as feature warping. However, features extracted by an encoder in feature warping cannot guarantee to accurately characterize the source identity and thus consequently produce a blur or lowfidelity image in an inevitable way.…”

Section: Introductionmentioning

confidence: 99%

“…It outputs 3D mesh (without clothes) rather than the layouts of joints and parts. Further, transformation flows can be easily calculated by matching the correspondences between two 3D triangulated meshes, which is more accurate and results in fewer misalignments than previous fitted affine matrix from keypoints [1,31]. Based on SMPL model and Liquid Warping Block (LWB), our method can be further extended into other tasks, including human appearance transfer and novel view synthesis for free and one model can handle these three tasks.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis

Liu

Piao

Min

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

274

263

View full text Add to dashboard Cite

We tackle the human motion imitation, appearance transfer, and novel view synthesis within a unified framework, which means that the model once being trained can be used to handle all these tasks. The existing taskspecific methods mainly use 2D keypoints (pose) to estimate the human body structure. However, they only expresses the position information with no abilities to characterize the personalized shape of the individual person and model the limbs rotations. In this paper, we propose to use a 3D body mesh recovery module to disentangle the pose and shape, which can not only model the joint location and rotation but also characterize the personalized body shape. To preserve the source information, such as texture, style, color, and face identity, we propose a Liquid Warping GAN with Liquid Warping Block (LWB) that propagates the source information in both image and feature spaces, and synthesizes an image with respect to the reference. Specifically, the source features are extracted by a denoising convolutional auto-encoder for characterizing the source identity well. Furthermore, our proposed method is able to support a more flexible warping from multiple sources. In addition, we build a new dataset, namely Impersonator (iPER) dataset, for the evaluation of human motion imitation, appearance transfer, and novel view synthesis. Extensive experiments demonstrate the effectiveness of our method in several aspects, such as robustness in occlusion case and preserving face identity, shape consistency and clothes details. All codes and datasets are available on https://svip-lab.github.io/ project/impersonator.html.

show abstract

Systematic survey on generative adversarial networks for brain tumor segmentation and classification

Kaur

Singh

Jindal

2023

Concurrency and Computation

View full text Add to dashboard Cite

SummaryBrain cancer is one of the leading diseases of death in the world caused due to unwanted proliferation of cells. If left untreated, this growth can spread into other human body parts. So early detection helps to enhance long‐term survival and reduce the mortality rate. A lot of research has been done in the field of brain tumors. There have been numerous previous studies conducted using traditional approaches. So far, few review studies are available on brain tumor segmentation and classification and moreover, they were quite limited in explaining the connections of Generative Adversarial Networks (GAN) for brain tumor segmentation and classification. This review mainly focuses on the GAN architectures for brain tumor segmentation and classification. Therefore, first, an overview of brain tumors, GAN‐based publications, approaches to brain tumor segmentation and classification, data sets, software platforms, and performance parameters, are presented. Finally, GAN applications and current trends are discussed. This thorough survey will aid researchers and beginners in the future in creating a better decision support system.

show abstract

Deformable GANs for Pose-Based Human Image Generation

Cited by 498 publications

References 23 publications

Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation

Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation

Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis

Systematic survey on generative adversarial networks for brain tumor segmentation and classification

Contact Info

Product

Resources

About