End-to-End Face Parsing via Interlinked Convolutional Neural Networks

Abstract

Face parsing is an important computer vision task that requires accurate pixel segmentation of facial parts (such as eyes, nose, mouth, etc.), providing a basis for further face analysis, modification, and other applications. In this paper, we introduce a simple, end-to-end face parsing framework: STN-aided iCNN (STN-iCNN), which extends interlinked Convolutional Neural Network (iCNN) by adding a Spatial Transformer Network (STN) between the two isolated stages. The STN-iCNN uses the STN to provide a trainable connection to the original two-stage iCNN pipeline, making end-to-end joint training possible. Moreover, as a by-product, STN also provides more precise cropped parts than the original cropper. Due to the two advantages, our approach significantly improves the accuracy of the original model.

AGRNet: Adaptive Graph Representation Learning and Reasoning for Face Parsing

“…Region-based parsing takes the scale discrepancy into account and predicts each component respectively, which is advantageous in capturing elaborate details [12], [13], [14], [15], [16]. Zhou et al present an interlinked CNN that takes multi-scale images as input and allows bidirectional information passing [15].…”

“…This method demonstrates high performance especially for hair segmentation [13]. Yin et al introduce the Spatial Transformer Network and build a training connection between traditional interlinked CNNs, which makes the end-to-end joint training process possible [14]. Nevertheless, this class of methods often neglect the correlation among components to characterize long range dependencies.…”

“…This however may neglect the scale discrepancy in different facial components, sometimes resulting in the lack of details. 2) Region-based parsing, which addresses the above problem by first predicting the bounding box and then acquiring the parsing map of each facial component individually [12], [13], [14], [15], [16]. However, there still exist limitations and challenges.…”

Edge-aware Graph Representation Learning and Reasoning for Face Parsing

“…We conduct experiments on the broadly acknowledged Helen dataset to demonstrate the superiority of the proposed model. To keep consistent with the previous works [6,4,38,5,33], we employ the overall F1 score to measure the performance, which is computed by combining the merged eyes, brows, nose and mouth categories. As Table 3 shows, Our model surpasses state-of-the-art methods and achieves 93.2% on this dataset.…”

“…The region-based methods have been recently proposed to model the facial components separately [4,5,6], and achieved state-of-the-art performance on the current benchmarks. However, these methods are based on the individual information within each region, and the correlation among regions is not exploited yet to capture long range dependencies.…”