Action Unit Detection by Learning the Deformation Coefficients of a 3D Morphable Model

Abstract

Facial Action Units (AUs) correspond to the deformation/contraction of individual facial muscles or their combinations. As such, each AU affects just a small portion of the face, with deformations that are asymmetric in many cases. Generating and analyzing AUs in 3D is particularly relevant for the potential applications it can enable. In this paper, we propose a solution for 3D AU detection and synthesis by developing on a newly defined 3D Morphable Model (3DMM) of the face. Differently from most of the 3DMMs existing in the literature, which mainly model global variations of the face and show limitations in adapting to local and asymmetric deformations, the proposed solution is specifically devised to cope with such difficult morphings. During a training phase, the deformation coefficients are learned that enable the 3DMM to deform to 3D target scans showing neutral and facial expression of the same individual, thus decoupling expression from identity deformations. Then, such deformation coefficients are used, on the one hand, to train an AU classifier, on the other, they can be applied to a 3D neutral scan to generate AU deformations in a subject-independent manner. The proposed approach for AU detection is validated on the Bosphorus dataset, reporting competitive results with respect to the state-of-the-art, even in a challenging cross-dataset setting. We further show the learned coefficients are general enough to synthesize realistic 3D face instances with AUs activation.

Facial Expression Recognition with Geometric Scattering on 3D Point Clouds

“…3D FER methods have succeeded in mining prominent geometrical details by SPD (Symmetric Positive Matrix) [ 15 ], conformal mapping [ 3 ], depth map [ 16 ], and recently with the prevalent statistical 3D Morphable Model (3DMM) [ 17 ] and point sets [ 11 ]. Among these, Refs.…”

The Florence 4D Facial Expression Dataset

“…Facial expressions play a primary role in interpersonal relations and are one fundamental way to convey our emotional state [7]. The automatic analysis of facial expressions focused first on images and videos, with rare examples using 3D data [1], [11], [26], [32]. These initial studies focused more on datasets with posed expressions [18], impersonated by actors.…”