Feature-Improving Generative Adversarial Network for Face Frontalization

Abstract

Face frontalization can boost the performance of face recognition methods and has made significant progress with the development of Generative Adversarial Networks (GANs). However, many GAN-based face frontalization methods still perform relatively weak on face recognition tasks under large face poses. In this paper, we propose Feature-Improving GAN (FI-GAN) for face frontalization, which aims to improve the recognition performance under large face poses. We assume that there is an inherent mapping between the frontal face and profile face, and their discrepancy in deep representation space can be estimated. The generation module of FI-GAN has a compact module named Feature-Mapping Block that helps to map the features of profile face images to the frontal space. Moreover, we produce a feature discriminator that can distinguish the features of profile face images from those of ground true frontal face images, which guide the generation module to provide high-quality features of profile faces. We conduct experiments on the MultiPIE, Labeled Faces in the Wild (LFW), and Celebrities in Frontal-Profile (CFP) databases. Our method is comparable to state-of-the-art methods under small poses and outperforms them on large pose face recognition.

Semi-Supervised Face Frontalization in the Wild

“…Frontal-Frontal Frontal-Profile Sengupta et al [3] 96.40 ± 0.69 84.91 ± 1.82 Sankarana et al [47] 96.93 ± 0.61 89.17 ± 2.35 LightCNN [42] 99.37 ± 0.30 91.56 ± 1.89 ResNet50 [43] 99.54 ± 0.31 94.25 ± 1.33 DR-GAN [12] 97.84 ± 0.79 93.41 ± 1.17 DR-GAN AM [19] 98.36 ± 0.75 93.89 ± 1.39 Chen et al [48] 98.67 ± 0.36 91.97 ± 1.70 PIM [17] 99.44 ± 0.36 93.10 ± 1.01 Peng et al [8] 98.67 ± − 93.76 ± − FI-GAN [32] 98…”

“…Their model first encode images by utilizing a pre-trained face expert network and then tried to recover photorealistic images from the extracted feature. Recently, Rong et al [32] proposed FI-GAN, aiming at improving the recognition performance under large face poses via a Feature-Mapping Block which maps the features of profile space to the frontal space. In most face frontalization methods, a large number of paired nonfrontal-frontal face images, typically from the MultiPIE dataset, are required to train the model in a fully supervised manner.…”

“…By contrast, most existing face frontalization methods are only trained on indoor face images (typically from Multi-PIE [20] dataset), which limits their generalization abilities in the unconstrained environment. [31] 93.62 ± 1.17 98.38 ± 0.06 HPEN [30] 96.25 ± 0.76 99.39 ± 0.02 LightCNN [42] 98.87 ± 0.61 99.69 ± 0.17 ResNet50 [43] 98.98 ± 0.52 99.79 ± 0.14 FF-GAN [13] 96.42 ± 0.89 99.45 ± 0.03 A3FCNN [15] 96.63 ± 0.99 99.29 ± 0.42 FI-GAN [32] 98. 30 We also conducted the face recognition experiment on the Multi-PIE dataset following the settings used with TP-GAN [14].…”

Robust Face Frontalization For Visual Speech Recognition

“…Some of the best performing DNN-based frontalization methods use CNN/GAN architectures, e.g. [54,27,49,59,57,43,58,55], which outperform CNNonly ones, e.g. [53].…”

Generative Adversarial Network for Multi Facial Attributes Translation

“…W Ith the improvement of generative models, such as the Generative Adversarial Networks (GANs) [1]- [3] and variational autoencoders (VAEs) [4], the processing of image-to-image translation have made considerable progresses. The translation includes photo style translation [5], [6], objects dyeing [7]- [9], and also facial synthesis [10]- [13].…”