Micro and Macro Facial Expression Recognition Using Advanced Local Motion Patterns

Abstract

In this paper, we develop a new method that recognizes facial expressions, on the basis of an innovative Local Motion Patterns (LMP) feature. The LMP feature analyzes locally the motion distribution in order to separate consistent mouvement patterns from noise. Indeed, facial motion extracted from the face is generally noisy and without specific processing, it can hardly cope with expression recognition requirements especially for micro-expressions. Direction and magnitude statistical profiles are jointly analyzed in order to filter out noise. This work presents three main contributions. The first one is the analysis of the face skin temporal elasticity and face deformations during expression. The second one is a unified approach for both macro and micro expression recognition leading the way to supporting a wide range of expression intensities. The third one is the step forward towards in-the-wild expression recognition, dealing with challenges such as various intensity and various expression activation patterns, illumination variations and small head pose variations. Our method outperforms state-of-the-art methods for micro expression recognition and positions itself among top-ranked state-of-the-art methods for macro expression recognition.

Facial expressions analysis under occlusions based on specificities of facial motion propagation

“…In order to characterize more precisely the movement, optical flow is particularly adapted and some features are derived from it. Recently, Allaert et al [1] proposed a descriptor called Local Motion Patterns (LMP) based on optical flow. It characterizes the facial movement by retaining only the main directions related to facial expressions, while avoiding motion discontinuities.…”

“…However, in presence of an important facial occlusion, the information to characterize the facial expression is almost completely lost or has a high probability of being noisy due to estimation errors. Recent approaches have proven the e↵ectiveness of optical flow in characterizing facial expressions [1]. Thanks to the physical properties of skin, descriptors based on movement seem adapted in the case of occlusion.…”

“…The first step generates various partial facial frameworks (a subset of facial regions), called configurations, including fewer regions than the initial facial framework. Inspired by [1], we consider a facial framework using regions laid out following the facial muscle scheme. For each configuration C j , the weighting algorithm evaluates the performance of the classification process using only the motion information contained in the regions R j composing C j .…”

“…Considering the weight evaluation in presence of Fig. 3: Heatmaps of the importance of regions per facial expression computed using LMP descriptor [1] on CK+ dataset [20]. See Section 6 for details.…”

Feature Selection on 2D and 3D Geometric Features to Improve Facial Expression Recognition

“…Focusing on the geometric approach and considering the type of data it is possible to extract features in 2D [ 14 , 15 ] and 3D [ 11 , 16 , 17 ]. Using a dynamic approach with sequence of images and employing advanced local motion patterns, it has been possible to recognize micro- and macro-expression in-the-wild in a unified framework [ 18 ].…”