Super-FAN: Integrated Facial Landmark Localization and Super-Resolution of Real-World Low Resolution Faces in Arbitrary Poses with GANs

Bulat, Adrian; Tzimiropoulos, Georgios

doi:10.1109/cvpr.2018.00019

Cited by 308 publications

(234 citation statements)

References 29 publications

(84 reference statements)

Supporting

Mentioning

234

Contrasting

Order By: Relevance

“…To our knowledge, the only method that reports face super-resolution results for real-world LR facial images is the very recent work of [28] which presents impressive qualitative results on more than 200 facial images taken from the Widerface dataset [1]. However, [28] is face-specific making use of facial landmarks for producing these results, rendering the approach inapplicable for other object categories for which landmarks are not available or landmark localization is not so effective. Contrary to many face super-resolution methods, the proposed pipeline is potentially applicable to other object categories.…”

Section: Closely Related Workmentioning

confidence: 99%

To Learn Image Super-Resolution, Use a GAN to Learn How to Do Image Degradation First

Bulat

Yang

Tzimiropoulos

2018

Lecture Notes in Computer Science

Self Cite

343

300

View full text Add to dashboard Cite

This paper is on image and face super-resolution. The vast majority of prior work for this problem focus on how to increase the resolution of low-resolution images which are artificially generated by simple bilinear down-sampling (or in a few cases by blurring followed by down-sampling). We show that such methods fail to produce good results when applied to real-world low-resolution, low quality images. To circumvent this problem, we propose a two-stage process which firstly trains a High-to-Low Generative Adversarial Network (GAN) to learn how to degrade and downsample high-resolution images requiring, during training, only unpaired high and low-resolution images. Once this is achieved, the output of this network is used to train a Low-to-High GAN for image super-resolution using this time paired low-and high-resolution images. Our main result is that this network can be now used to effectively increase the quality of real-world low-resolution images. We have applied the proposed pipeline for the problem of face super-resolution where we report large improvement over baselines and prior work although the proposed method is potentially applicable to other object categories.

show abstract

Section: Closely Related Workmentioning

confidence: 99%

To Learn Image Super-Resolution, Use a GAN to Learn How to Do Image Degradation First

Bulat

Yang

Tzimiropoulos

2018

Lecture Notes in Computer Science

Self Cite

343

300

View full text Add to dashboard Cite

show abstract

“…Taking X domain for example. We adopt the architecture for our structure encoder E g x from Stack-Hourglass network [31] which have shown impressive results for landmark localisation task [7,3]. For the mapping from g x tox (E c x and D x with skip-connection), we use the UNet architecture [33] provided by [52].…”

Section: Methodsmentioning

confidence: 99%

TransGaGa: Geometry-Aware Unsupervised Image-To-Image Translation

Wu¹,

Cao

Cheng³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

121

View full text Add to dashboard Cite

Figure 1: We propose a geometry-aware framework for unsupervised image-to-image translation, which is robust to arbitrary shape variations between domains. We show the results of both near-rigid and non-rigid objects. (left) Cow and cheetah rendered from CAD models. (center) Cat and human face from in-the-wild datasets. (right) Horse and giraffe from Flickr. AbstractUnsupervised image-to-image translation aims at learning a mapping between two visual domains. However, learning a translation across large geometry variations always ends up with failure. In this work, we present a novel disentangle-and-translate framework to tackle the complex objects image-to-image translation task. Instead of learning the mapping on the image space directly, we disentangle image space into a Cartesian product of the appearance and the geometry latent spaces. Specifically, we first introduce a geometry prior loss and a conditional VAE loss to encourage the network to learn independent but complementary representations. The translation is then built on appearance and geometry space separately. Extensive experiments demonstrate the superior performance of our method to other state-of-the-art approaches, especially in the challenging near-rigid and non-rigid objects translation tasks. In addition, by taking different exemplars as the appearance references, our method also supports multimodal translation. Project page: https://wywu.github. io/projects/TGaGa/TGaGa.html

show abstract

“…Structure information could efficiently assist deblurring. Prior knowledge, especially facial structure [15,2,35], has been proven to be an effective face prior in corresponding tasks such as super-resolution and deblurring. However, as mentioned in [35], these methods fail when the input face images are not well aligned, e.g.…”

Section: Motion Deblurringmentioning

confidence: 99%

FAB: A Robust Facial Landmark Detection Framework for Motion-Blurred Videos

Sun

Liu

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Recently, facial landmark detection algorithms have achieved remarkable performance on static images. However, these algorithms are neither accurate nor stable in motion-blurred videos. The missing of structure information makes it difficult for state-of-the-art facial landmark detection algorithms to yield good results.In this paper, we propose a framework named FAB that takes advantage of structure consistency in the temporal dimension for facial landmark detection in motionblurred videos. A structure predictor is proposed to predict the missing face structural information temporally, which serves as a geometry prior. This allows our framework to work as a virtuous circle. On one hand, the geometry prior helps our structure-aware deblurring network generates high quality deblurred images which lead to better landmark detection results. On the other hand, better landmark detection results help structure predictor generate better geometry prior for the next frame. Moreover, it is a flexible video-based framework that can incorporate any static image-based methods to provide a performance boost on video datasets. Extensive experiments on Blurred-300VW, the proposed Realworld Motion Blur (RWMB) datasets and 300VW demonstrate the superior performance to the state-of-the-art methods. Datasets and models will be publicly available at https://keqiangsun.github.io/projects/FAB/FAB.html.

show abstract

Super-FAN: Integrated Facial Landmark Localization and Super-Resolution of Real-World Low Resolution Faces in Arbitrary Poses with GANs

Cited by 308 publications

References 29 publications

To Learn Image Super-Resolution, Use a GAN to Learn How to Do Image Degradation First

To Learn Image Super-Resolution, Use a GAN to Learn How to Do Image Degradation First

TransGaGa: Geometry-Aware Unsupervised Image-To-Image Translation

FAB: A Robust Facial Landmark Detection Framework for Motion-Blurred Videos

Contact Info

Product

Resources

About