GANimation: Anatomically-Aware Facial Animation from a Single Image

Pumarola, Albert; Agudo, Antonio; Martı́nez, Aleix M.; Sanfeliu, Alberto; Moreno-Noguer, Francesc

doi:10.1007/978-3-030-01249-6_50

Cited by 478 publications

(539 citation statements)

References 28 publications

(66 reference statements)

Supporting

Mentioning

537

Contrasting

Unclassified

Order By: Relevance

“…For each sample, from top to bottom, they are images generated by AcGANs, IPCGANs, and CAAE. The input age lines in [11][12][13][14][15][16][17][18][19][20] age group and the numbers above the images are the corresponding target age.…”

Section: -30mentioning

confidence: 99%

“…Inspired by the success of attention mechanism in imageto-image translation [15], in this paper, we propose an Attention Conditional GANs (AcGANs) to tackle these issues mentioned-above. Specifically, the proposed AcGANs consists of a generator G and a discriminator D. The generator G receives an input image and a target age code and the output of the generator contains an attention mask and a color mask.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Look Globally, Age Locally: Face Aging With an Attention Mechanism

Zhu

Huang

Shan

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Face aging is of great importance for cross-age recognition and entertainment-related applications. Recently, conditional generative adversarial networks (cGANs) have achieved impressive results for face aging. Existing cGANs-based methods usually require a pixel-wise loss to keep the identity and background consistent. However, minimizing the pixel-wise loss between the input and synthesized images likely resulting in a ghosted or blurry face. To address this deficiency, this paper introduces an Attention Conditional GANs (AcGANs) approach for face aging, which utilizes attention mechanism to only alert the regions relevant to face aging. In doing so, the synthesized face can well preserve the background information and personal identity without using the pixel-wise loss, and the ghost artifacts and blurriness can be significantly reduced. Based on the benchmarked dataset Morph, both qualitative and quantitative experiment results demonstrate superior performance over existing algorithms in terms of image quality, personal identity, and age accuracy.

show abstract

Section: -30mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Look Globally, Age Locally: Face Aging With an Attention Mechanism

Zhu

Huang

Shan

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…To alleviate this challenge, we introduce a switching architecture ( Fig. 2 (c)) that adapts the blurring strength using a mask, inspired by the recent success of masks for adjusting generative components (e.g., foreground and background [79] and retention and translation [68]). Concretely, the final output of the kernel generator is calculated by…”

Section: Blur Robust Gan: Br-ganmentioning

confidence: 99%

Label-Noise Robust Generative Adversarial Networks

Kaneko

Ushiku

Harada

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Recently, generative adversarial networks (GANs), which learn data distributions through adversarial training, have gained special attention owing to their high image reproduction ability. However, one limitation of standard GANs is that they recreate training images faithfully despite image degradation characteristics such as blur, noise, and compression. To remedy this, we address the problem of blur, noise, and compression robust image generation. Our objective is to learn a non-degraded image generator directly from degraded images without prior knowledge of image degradation. The recently proposed noise robust GAN (NR-GAN) already provides a solution to the problem of noise degradation. Therefore, we first focus on blur and compression degradations. We propose blur robust GAN (BR-GAN) and compression robust GAN (CR-GAN), which learn a kernel generator and quality factor generator, respectively, with non-degraded image generators. Owing to the irreversible blur and compression characteristics, adjusting their strengths is non-trivial. Therefore, we incorporate switching architectures that can adapt the strengths in a data-driven manner. Based on BR-GAN, NR-GAN, and CR-GAN, we further propose blur, noise, and compression robust GAN (BNCR-GAN), which unifies these three models into a single model with additionally introduced adaptive consistency losses that suppress the uncertainty caused by the combination. We provide benchmark scores through large-scale comparative studies on CIFAR-10 and a generality analysis on FFHQ dataset.1 More precisely, to reduce this cost, unpaired learning methods [53,50] were devised.These methods, however, still require separate collection of non-degraded and degraded images. As an alternative, self-supervised learning methods [45, 2, 48] were also proposed; however, their application is limited to denoising.Blur, Noise, and Compression Robust Generative Adversarial Networks 3 Motivated by these existing studies, we address the following problem: "How can we learn a non-degraded image generator directly from degraded images without knowing the details of image degradation? " Particularly, in this work, we focus on three representative types of image degradation, i.e., blur, noise, and compression, and call this problem blur, noise, and compression robust image generation. As an example, we demonstrate a solution using our model in Fig. 1(c). Although the same training images ( Fig. 1(a)) are used between baseline GAN (Fig. 1(b)) and proposed blur, noise, and compression robust GAN (BNCR-GAN ) (Fig. 1(c)), BNCR-GAN succeeds in generating non-degraded images, whereas baseline GAN fails to do so.The recently proposed noise robust GAN (NR-GAN) [31] has shown promising results when its application is restricted to noise degradation. Hence, in this work, we first focus on the remaining two kinds of degradation, i.e., blur and compression, and handle all types of degradation as an advanced problem.To obtain robustness in blur and compression degradations, we first propose two variants of ...

show abstract

“…Nevertheless, successful generation of plausible samples on extreme face geomorphing and complex uncontrolled datasets remain elusive goals for these methods due to their unstable training procedures and environmental constraints. The current state-ofthe-art [32] in RaFD [26] face expression synthesis achieves a Fréchet Inception Distance (FID) [15] of 34, still leaving a large gap towards real data even in a controlled environment.…”

Section: Introductionmentioning

confidence: 99%

Make a Face: Towards Arbitrary High Fidelity Face Manipulation

Qian

Lin

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

InputFace Manipulation Results of Our Model Figure 1: Face manipulation results on in-the-wild samples via transferring knowledge learned from the CelebA dataset. The first column shows input images and the remainders are images generated by AF-VAE with target expression/rotation boundary maps as the condition. Note that the model is fine-tuned with movie clip frames from YouTube of 256 × 256 resolution. All the generated poses are unseen before. AbstractRecent studies have shown remarkable success in face manipulation task with the advance of GANs and VAEs paradigms, but the outputs are sometimes limited to lowresolution and lack of diversity.In this work, we propose Additive Focal Variational Auto-encoder (AF-VAE), a novel approach that can arbitrarily manipulate high-resolution face images using a simple yet effective model and only weak supervision of reconstruction and KL divergence losses. First, a novel additive Gaussian Mixture assumption is introduced with an unsupervised clustering mechanism in the structural latent 1 Work done during an internship at SenseTime Research. space, which endows better disentanglement and boosts multi-modal representation with external memory. Second, to improve the perceptual quality of synthesized results, two simple strategies in architecture design are further tailored and discussed on the behavior of Human Visual System (HVS) for the first time, allowing for fine control over the model complexity and sample quality. Human opinion studies and new state-of-the-art Inception Score (IS) / Fréchet Inception Distance (FID) demonstrate the superiority of our approach over existing algorithms, advancing both the fidelity and extremity of face manipulation task.

show abstract

GANimation: Anatomically-Aware Facial Animation from a Single Image

Cited by 478 publications

References 28 publications

Look Globally, Age Locally: Face Aging With an Attention Mechanism

Look Globally, Age Locally: Face Aging With an Attention Mechanism

Label-Noise Robust Generative Adversarial Networks

Make a Face: Towards Arbitrary High Fidelity Face Manipulation

Contact Info

Product

Resources

About