Stacked Dense U-Nets with Dual Transformers for Robust Face Alignment

Abstract

Facial landmark localisation in images captured in-the-wild is an important and challenging problem. The current state-of-the-art revolves around certain kinds of Deep Convolutional Neural Networks (DCNNs) such as stacked U-Nets and Hourglass networks. In this work, we innovatively propose stacked dense U-Nets for this task. We design a novel scale aggregation network topology structure and a channel aggregation building block to improve the model's capacity without sacrificing the computational complexity and model size. With the assistance of deformable convolutions inside the stacked dense U-Nets and coherent loss for outside data transformation, our model obtains the ability to be spatially invariant to arbitrary input face images. Extensive experiments on many in-the-wild datasets, validate the robustness of the proposed method under extreme poses, exaggerated expressions and heavy occlusions. Finally, we show that accurate 3D face alignment can assist pose-invariant face recognition where we achieve a new stateof-the-art accuracy on CFP-FP (98.514%).

Infrared-Fused Vision-Based Thermoregulation Performance Estimation for Personal Thermal Comfort-Driven HVAC System Controls

“…Physiological measurement-based techniques could address the short-term comfort requirements, as they infer occupant thermal comfort by monitoring biological processesspecifically, thermoregulation system performance (e.g., vasodilation, vasoconstriction, shivering, and sweating), relevant physiological responses (e.g., heart rate, and electroencephalograph), and relevant physical responses (e.g., changes in posture) [13,17,18]. The thermoregulation system regulates body temperature by vasodilation (widening blood vessels) when faced with hot stresses and vasoconstriction (constricting blood vessels) when faced with cold stresses [19].…”

“…A stacked hourglass network enables inferencing by first processing features down to low resolutions before the network begins upsampling and combining features across scales to produce a set of predictions [15]. Algorithm Candidate 2 was developed by InsightFace [18,39]. They employ a stacked hourglass network and can produce 68 unique coordinates through channel aggregation residual blocks rather than HPM residual blocks.…”

“…They employ a stacked hourglass network and can produce 68 unique coordinates through channel aggregation residual blocks rather than HPM residual blocks. Although they share similar structures, the difference lies in the number of weight layers used for computing the output [18]. Algorithm Candidate 3 was an implementation of face alignment developed by Iván de Paz Centeno (i.e., FaceNet) using multitask cascaded convolutional neural network first explored by K. Zhang et al [40].…”

Head2HeadFS: Video-based Head Reenactment with Few-shot Learning

“…We adopt here a fast and robust sparse landmark-based method that capitalises on the rich temporal information in videos while performing the reconstruction. We rely on the fact that state-ofthe-art facial alignment methods are quite robust and accurate, and use the method of [18] to extract 68 landmarks from each frame. While carrying out the 3D reconstruction, we postulate scaled orthographic projection (SOP) and assume that in each video the identity parameters s i t are fixed (yet unknown) throughout the entire video, letting however the expression parameters s e t as well as the camera parameters (scale and 3D pose) to differ among frames.…”

Head2Head: Video-based Neural Head Synthesis

“…illumination and reflectance) and solving a highly ill-posed problem. On the other hand, our facial reconstruction and tracking stage is a sparse-landmarks-based fast approach, which requires only 68 facial landmarks extracted by [44], as well as the frame sequence. Thanks to our novel video rendering framework, the facial representation extracted by our face tracker encapsulates adequate information for synthesising photo-realistic and temporally smooth videos, removing the need for more elaborate and slower 3D facial reconstruction and tracking techniques.…”