Style Aggregated Network for Facial Landmark Detection

Abstract

Recent advances in facial landmark detection achieve success by learning discriminative features from rich deformation of face shapes and poses. Besides the variance of faces themselves, the intrinsic variance of image styles, e.g., grayscale vs. color images, light vs. dark, intense vs. dull, and so on, has constantly been overlooked. This issue becomes inevitable as increasing web images are collected from various sources for training neural networks. In this work, we propose a style-aggregated approach to deal with the large intrinsic variance of image styles for facial landmark detection. Our method transforms original face images to style-aggregated images by a generative adversarial module. The proposed scheme uses the style-aggregated image to maintain face images that are more robust to environmental changes. Then the original face images accompanying with style-aggregated ones play a duet to train a landmark detector which is complementary to each other. In this way, for each face, our method takes two images as input, i.e., one in its original style and the other in the aggregated style. In experiments, we observe that the large variance of image styles would degenerate the performance of facial landmark detectors. Moreover, we show the robustness of our method to the large variance of image styles by comparing to a variant of our approach, in which the generative adversarial module is removed, and no style-aggregated images are used. Our approach is demonstrated to perform well when compared with state-of-the-art algorithms on benchmark datasets AFLW and 300-W. Code is publicly available on GitHub: https://github.com/D-X-Y/SAN

DeCaFA: Deep Convolutional Cascade for Face Alignment in the Wild

“…CPM+SBR [5] employs landmark registration to regularize training. SAN [4] uses adversarial networks to convert images from different styles to an aggregated style, upon which regression is performed. This aggregated style space thus serve as an intermediate representation that is more convenient for training.…”

FAB: A Robust Facial Landmark Detection Framework for Motion-Blurred Videos

“…The classic model-based methods ASMs [27], AAMs [7,17,23,32], CLMs [19,33], ESR [3], SDM [41], CFSS [46], and deep convolutional neural wetwork methods, e.g. TCDCN [45], FAN [1], DSRN [26], RFLD [25], SAN [8], LAB [38] have obtained increasingly excellent performance in static images under different poses, light conditions, expressions, etc. However, few works in the literature of face alignment pay attention to the motion blur.…”

“…This experiment is conducted to demonstrate that our algorithm is capable to handle artificial motion blur, and accurately detect facial landmarks. We re-implemented several typical state-of-the-art algorithms for comparison, including RFLD [25], SAN [8] and LAB [38]. For a fair comparison, we made great efforts to reproduce the comparable results reported in their papers.…”

“…Figure 4 shows the CED curve of all these excellent algorithms. Note that SAN [8] and LAB [38] are rather robust in image-based facial landmark detection. However, when confronting the severe motion blur, they fail to yield reliable and robust facial landmarks due to the destruction of the structure information.…”

“…face recognition, face verification and face augmented reality. Previous researches [41,45,46,39,8,9,38,25] mainly Figure 1: The first column is the frames of Blurred-300VW. In the second column, the results of state-of-the-art facial landmark detection algorithm are given.…”