3D Visual passcode: Speech-driven 3D facial dynamics for behaviometrics

Abstract

Face biometrics have achieved remarkable performance over the past decades, but unexpected spoofing of the static faces poses a threat to information security. There is an increasing demand for stable and discriminative biological modalities which are hard to be mimicked and deceived. Speech-driven 3D facial motion is a distinctive and measurable behavior-signature that is promising for biometrics. In this paper, we propose a novel 3D behaviometrics framework based on a "3D visual passcode" derived from speech-driven 3D facial dynamics. The 3D facial dynamics are jointly represented by 3D-keypoint-based measurements and 3D shape patch features, extracted from both static and speech-driven dynamic regions. An ensemble of subject-specific classifiers are then trained over selected discriminative features, which allows for a discriminant speech-driven 3D facial dynamics representation. We construct the first publicly available Speech-driven 3D Facial Motion dataset (S3DFM) that includes 2D-3D face video plus audio samples from 77 participants. The experimental results on the S3DFM show that the proposed pipeline achieves a face identification rate of 96.1%. Detailed discussions are presented, concerning anti-spoofing, head pose variation, video frame rate, and applicability cases. We also give comparison with other baselines on "deep" and "shallow" 2D face features.

Attention-Driven Cropping for Very High Resolution Facial Landmark Detection

“…• We combine attention driven cropping, introduced by [ [48,49,4]. These datasets contain annotations for 68 facial landmarks.…”

The 4D Space-Time Dimensions of Facial Perception

“…Facial movement can be modeled as rigid and non-rigid motion (for review see Smeets et al, 2012;Chrysos et al, 2018). Evidence suggests anatomically accurate rigid and non-rigid facial movements are essential to facial perception (Trautmann et al, 2009;Johnston et al, 2013) and is achievable computationally (Zhang et al, 2003;Zhang and Fisher, 2019). Deformable facial modeling, relates to movement we produce through voluntary, striated musculature or facial muscles.…”

“…It is worth highlighting that a selection of 3D and 4D facial databases, have been developed with specific applications across disciplines. Facial identification and recognition studies, commonly referred to as biometrics, may benefit from the databases developed by Benedikt et al (2010) ; Feng et al (2014) , and Zhang and Fisher (2019) . A relatively large corpus of datasets has been developed to present stimuli as three-dimensional forms or structures on-screen (built with breadth, height and depth), as static emotional expressions ( FIDENTIS-3D ; Urbanová et al, 2018 ; Bosphorus ; Savran et al, 2008 ; BU-3DFE ; Yin et al, 2006 ; ICT-3DRFE ; Stratou et al, 2011 ; ND-2006 ; Faltemier et al, 2008 ; Casia 3D ; Zhong et al, 2007 ; The York 3D ; Heseltine et al, 2008 ; GAVDB ; Moreno, 2004 ; Texas 3DFRD ; Gupta et al, 2010 ).…”

3D Lip Event Detection via Interframe Motion Divergence at Multiple Temporal Resolutions

“…Speaking is a spontaneous behavior involving multiple biological modalities, including voice, visual speech [10], dynamic face motions [20], etc. There are many applications related with speaking, e.g.…”

“…There are many applications related with speaking, e.g. speech recognition, lipreading [8], or identity recognition [10,20]. The lip is a dominant dynamic facial unit when speaking, and the visual lip acts as an important counterpart to the audio information.…”