Robust facial landmark detection by cross-order cross-semantic deep network

Abstract

Recently, convolutional neural networks (CNNs)-based facial landmark detection methods have achieved great success. However, most of existing CNN-based facial landmark detection methods have not attempted to activate multiple correlated facial parts and learn different semantic features from them that they can not accurately model the relationships among the local details and can not fully explore more discriminative and fine semantic features, thus they suffer from partial occlusions and large pose variations. To address these problems, we propose a cross-order cross-semantic deep network (CCDN) to boost the semantic features learning for robust facial landmark detection. Specifically, a crossorder two-squeeze multi-excitation (CTM) module is proposed to introduce the cross-order channel correlations for more discriminative representations learning and multiple attention-specific part activation. Moreover, a novel cross-order cross-semantic (COCS) regularizer is designed to drive the network to learn cross-order cross-semantic features from different activation for facial landmark detection. It is interesting to show that by integrating the CTM module and COCS regularizer, the proposed CCDN can effectively activate and learn more Fully documented templates are available in the elsarticle package on CTAN.

Robust facial landmark detection by probability‐guided hourglass network

“…Table 2 shows the NMEs of the various algorithms, where the mean error is normalized by inter‐pupil distance and inter‐ocular distance. These data are from the original paper, including RCPR [12], LBF [29], TCDCN [30], CFSS [31], MDM [17], DCFE [18], SAN [32], LAB [13], Wing [20], LUVLi [24], FCDN [7], SHN‐GCN [8], 2‐SCRM [15], ASMNet [21], CCNN [25], PIPNet [33] and MAttHG [9].…”

“…To evaluate the robustness of PGHN against heavy occlusions, we assess our model on the COFW dataset. Table 3 shows the results of PGHN compared with the state‐of‐the‐art methods, including RCPR [12], LBF [29], TCDCN [30], CFSS [31], LAB [13], Wing [20], RWing [19], FCDN [7], SHN‐GCN [8], CCNN [25] and MAttHG [9].…”

“…To evaluate the robustness of our model against various situations, we evaluate PGHN on WFLW dataset. We compare with several state‐of‐the‐art methods, including LBF [29], CFSS [31], LAB [13], Wing [20], RWing [19], FCDN [7], ASMNet [21], PIPNet [33], MAttHG [9], MMDN [28] and DSCN [10].…”

“…Wan et al. [7] improve the robustness of the hourglass network by integrating multiple multi‐order spatial correlation modules and a multi‐order channel correlation module into the SHN, which aims to generate more efficient and accurate heatmaps. Zhang et al.…”

LocalEyenet: Deep Attention Framework for Localization of Eyes

“…Researchers have recently proposed a semi-supervised method [21], called self-calibrated pose attention network (SCPAN), that computes Boundary-Aware Landmark Intensity (BALI) fields corresponding to a boundary and the landmarks closest to the boundary. They have also extended their work by proposing an implicit multiorder correlating geometry-aware (IMCG) model [22] [23], that uses spatial and channel correlations to attain the local as well as global features.…”