Can Hierarchical Transformers Learn Facial Geometry?

Abstract

Human faces are a core part of our identity and expression, and thus, understanding facial geometry is key to capturing this information. Automated systems that seek to make use of this information must have a way of modeling facial features in a way that makes them accessible. Hierarchical, multi-level architectures have the capability of capturing the different resolutions of representation involved. In this work, we propose using a hierarchical transformer architecture as a means of capturing a robust representation of facial geometry. We further demonstrate the versatility of our approach by using this transformer as a backbone to support three facial representation problems: face anti-spoofing, facial expression representation, and deepfake detection. The combination of effective fine-grained details alongside global attention representations makes this architecture an excellent candidate for these facial representation problems. We conduct numerous experiments first showcasing the ability of our approach to address common issues in facial modeling (pose, occlusions, and background variation) and capture facial symmetry, then demonstrating its effectiveness on three supplemental tasks.

Computer Vision in Human Analysis: From Face and Body to Clothes

“…Another topic investigated in this Special Issue is human face analysis. In particular, the published papers address different topics, including the problem of machine interaction using voice commands and facial movements [ 6 ], 3D face and body geometry reconstruction [ 7 , 8 ], and dyadic interaction analysis based on facial expressions [ 9 ]. Focusing on eye images, Gibertoni et al [ 10 ] propose a system to automatically classify the eye status in images acquired through an ophthalmic tool.…”