To Frontalize or Not to Frontalize: Do We Really Need Elaborate Pre-processing to Improve Face Recognition?

Banerjee, S.; Brogan, Joel; Krizaj, Janez; Bharati, Aparna; Webster, Brandon Richard; Flynn, Patrick J.; Scheirer, Walter J.

doi:10.1109/wacv.2018.00009

Cited by 25 publications

(37 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Domain adaptation tools help to move or transfer knowledge between different domains. In the specific case of face recognition, face-frontalisation methods (methods that convert any face to its frontal view, allowing us to eliminate the pose bias) [24][25][26] represent a way of domain adaptation to work just with frontal faces. Since we could define a domain like the one that represents a certain dataset, it is easy to understand how these approaches help us to deal with the problem.…”

Section: Nature Of Dataset Biasmentioning

confidence: 99%

Dataset bias exposed in face verification

et al. 2019

View full text Add to dashboard Cite

Most facial verification methods assume that training and testing sets contain independent and identically distributed samples, although, in many real applications, this assumption does not hold. Whenever gathering a representative dataset in the target domain is unfeasible, it is necessary to choose one of the already available (source domain) datasets. Here, a study was performed over the differences among six public datasets, and how this impacts on the performance of the learned methods. In the considered scenario of mobile devices, the individual of interest is enrolled using a few facial images taken in the operational domain, while training impostors are drawn from one of the public available datasets. This work tried to shed light on the inherent differences among the datasets, and potential harms that should be considered when they are combined for training and testing. Results indicate that a drop in performance occurs whenever training and testing are done on different datasets compared to the case of using the same dataset in both phases. However, the decay strongly depends on the kind of features. Besides, the representation of samples in the feature space reveals insights into what extent bias is an endogenous or an exogenous factor.

show abstract

Section: Nature Of Dataset Biasmentioning

confidence: 99%

Dataset bias exposed in face verification

et al. 2019

View full text Add to dashboard Cite

show abstract

“…reported a 1% recognition accuracy improvement on the LFW dataset when alignment was adopted in the testing stage [19]. It has also been shown that in terms of the recognition accuracy, 3D frontalization does not show significant improvements over simple 2D alignments [1]. Therefore, we will focus on the 2D image alignment only in this paper.…”

Section: Related Workmentioning

confidence: 99%

“…Suppose that for the i th target point p t i = (x t i , y t i , 1) in the output image, a grid generator generates its source coordinates (x s i , y s i , 1) in the input image according to transformation parameters. For the projective transformation, such a process can be expressed by (1) in which A to H are eight transformation parameters and z s i = Gx t i + Hy t i + 1.…”

Section: Spatial Transformer Networkmentioning

confidence: 99%

“…extracting facial features for recognition, (4) deciding personal identity based on the likelihood between feature templates. Recent research shows that while 3D alignment may seem superior over 2D alignment, it shows no significant advantage in terms of recognition accuracy especially when CNNs are used to extract facial features [1]. There are two major concerns regarding such a framework in which face alignment and facial feature extraction are performed independently.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Toward End-to-End Face Recognition Through Alignment Learning

Zhong

Chen

Huang

2017

IEEE Signal Process. Lett.

View full text Add to dashboard Cite

Plenty of effective methods have been proposed for face recognition during the past decade. Although these methods differ essentially in many aspects, a common practice of them is to specifically align the facial area based on the prior knowledge of human face structure before feature extraction. In most systems, the face alignment module is implemented independently. This has actually caused difficulties in the designing and training of end-to-end face recognition models. In this paper we study the possibility of alignment learning in end-to-end face recognition, in which neither prior knowledge on facial landmarks nor artificially defined geometric transformations are required. Specifically, spatial transformer layers are inserted in front of the feature extraction layers in a Convolutional Neural Network (CNN) for face recognition. Only human identity clues are used for driving the neural network to automatically learn the most suitable geometric transformation and the most appropriate facial area for the recognition task. To ensure reproducibility, our model is trained purely on the publicly available CASIA-WebFace dataset, and is tested on the Labeled Face in the Wild (LFW) dataset. We have achieved a verification accuracy of 99.08% which is comparable to state-of-the-art single model based methods.

show abstract

“…This frontalization process is also called pose normalization. AbS methods can be categorised as a) 3D methods (AbS-3D) [18], [19], [20], [21], [22], [23], [24], [25], [26] and b) 2D methods (AbS-2D) [27], [28], [29], [30], [31]. AbS-3D methods fit a 3D model, typically a 3D Morphable Model (3DMM) [32], to an input face image with arbitrary pose.…”

Section: Introductionmentioning

confidence: 99%

Face Frontalization Using an Appearance-Flow-Based Convolutional Neural Network

Chen

Wang

Hu³

et al. 2019

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Facial pose variation is one of the major factors making face recognition (FR) a challenging task. One popular solution is to convert non-frontal faces to frontal ones on which face recognition is performed. Rotating faces causes the facial pixel value changes. Therefore, existing CNN-based methods learn to synthesize frontal faces in color space. However, this learning problem in color space is highly non-linear, causing the synthetic frontal faces to lose fine facial textures. In this work, we take the view that the nonfrontal-frontal pixel changes are essentially caused by geometric transformations (rotation, translation, etc) in space. Therefore, we aim to learn the nonfrontal-frontal facial conversion in spatial domain rather than the color domain to ease the learning task. To this end, we propose an Appearance Flow based Face Frontalization Convolutional Neural Network (A3F-CNN). Specifically, A3F-CNN learns to establish the dense correspondence between the non-frontal and frontal faces. Once the correspondence is built, frontal faces are synthesized by explicitly 'moving' pixels from the non-frontal one. In this way, the synthetic frontal faces can preserve fine facial textures. To improve the convergence of training, an appearance flow guided learning strategy is proposed. In addition, GAN loss is applied to achieve a more photorealistic face and a face mirroring method is introduced to handle the self-occlusion problem. Extensive experiments are conducted on face synthesis and pose invariant face recognition. Results show that our method can synthesize more photorealistic faces than existing methods in both controlled and uncontrolled lighting environments. Moreover, we achieve very competitive face recognition performance on the Multi-PIE, LFW and IJB-A databases.

show abstract

To Frontalize or Not to Frontalize: Do We Really Need Elaborate Pre-processing to Improve Face Recognition?

Cited by 25 publications

References 51 publications

Dataset bias exposed in face verification

Dataset bias exposed in face verification

Toward End-to-End Face Recognition Through Alignment Learning

Face Frontalization Using an Appearance-Flow-Based Convolutional Neural Network

Contact Info

Product

Resources

About