FH-GAN: Face Hallucination and Recognition Using Generative Adversarial Network

Abstract

There are many factors affecting visual face recognition, such as low resolution images, aging, illumination and pose variance, etc. One of the most important problem is low resolution face images which can result in bad performance on face recognition. Most of the general face recognition algorithms usually assume a sufficient resolution for the face images. However, in practice many applications often do not have sufficient image resolutions. The modern face hallucination models demonstrate reasonable performance to reconstruct high-resolution images from its corresponding low resolution images. However, they do not consider identity level information during hallucination which directly affects results of the recognition of low resolution faces. To address this issue, we propose a Face Hallucination Generative Adversarial Network (FH-GAN) which improves the quality of low resolution face images and accurately recognize those low quality images. Concretely, we make the following contributions: 1) we propose FH-GAN network, an end-to-end system, that improves both face hallucination and face recognition simultaneously. The novelty of this proposed network depends on incorporating identity information in a GAN-based face hallucination algorithm via combining a face recognition network for identity preserving. 2) We also propose a new face hallucination network, namely Dense Sparse Network (DSNet), which improves upon the state-of-art in face hallucination. 3) We demonstrate benefits of training the face recognition and GAN-based DSNet jointly by reporting good result on face hallucination and recognition.

Deep Learning-based Face Super-Resolution: A Survey

“…Simple Identity Loss: The introduction of identity information aims to maintain identity consistency between SR and HR. To this end, a commonly used design is to define identity-based loss, e.g., deep joint face hallucination and recognition (DJFHR) [159], super-identity convolutional neural network (SICNN) [188], face hallucination generative adversarial network (FH-GAN) [8], optimizing super resolution for face recognition (FRSRResNet) [1], WaveSRGAN [60], low-resolution face recognition based on identity-preserved face hallucination (SRNet) [82], identity-preserving face hallucination via deep reinforcement learning (IPFH) [30] and identity-preserving face super-resolution (IPFSR) [16], ID preserving face super-resolution generative adversarial networks [89], cascaded super-resolution and identity priors (C-SRIP) [48] and facial semantic attribute transformation and self-attentive structure enhancement [93]. The frameworks of these methods consist of two main components: super-resolution model that generates SR results, and pre-trained face recognition Designing supervised pixel-wise GAN with identity embedding discriminator for face super-resolution Identity embedding discriminator network (FRN) for identity preserving, probably an additional discriminator.…”

“…Towards F , when F is 1, the loss is used in WaveSRGAN [60] and IPFSR [16], F is 2 in FH-GAN [8] and FRSRResNet [1], or…”

“…In the Table 8, we present whether the network includes FRN, and how these methods take advantage of identity information in detail. In essence, SICNN [188], FH-GAN [8], IPFSR [16] all take advantage of the distance between FRN(I HR ) and FRN(I SR ) as identity loss, although some differences between them, e.g., MSE loss or L 1 loss. Compared with them, C-SRIP's identity loss measures cross entropy distance between features of the residual map.…”

Small Object Recognition Using a Spatio-Temporal Neural Network

“…The attention mechanism is also adopted into many works and the main ideas include temporal attention [18] and spatial attention [19]. In some other classical computer vision problems as face recognition, models [20,21] showed a relatively poor performance on low-resolution dataset.…”

Transition from Holistic to Deep learning Face Recognition Methods

“…WebFace data Set is used to improve training of network and test. The paper proposes face hallucination (FH) and recognition system [20] using generative adversarial network in which face hallucination reconstruct a high resolution image from a low resolution image and recognition can be accurately done by the restricted image. The research proposes two models, first FH-GAN network which has improved face hallucination and recognition together.…”