Towards macro- and micro-expression spotting in video using strain patterns

Abstract

This paper presents a novel method for automatic spotting (temporal segmentation) of facial expressions in long videos comprising of continuous and changing expressions. The method utilizes the strain impacted on the facial skin due to the non-rigid motion caused during expressions. The strain magnitude is calculated using the central difference method over the robust and dense optical flow field of each subjects face. Testing has been done on 2 datasets (which includes 100 macro-expressions) and promising results have been obtained. The method is robust to several common drawbacks found in automatic facial expression segmentation including moderate in-plane and out-ofplane motion. Additionally, the method has also been modified to work with videos containing microexpressions. Micro-expressions are detected utilizing their smaller spatial and temporal extent. A subject's face is divided in to sub-regions (mouth, cheeks, forehead, and eyes) and facial strain is calculated for each of these regions. Strain patterns in individual regions are used to identify subtle changes which facilitate the detection of micro-expressions.

Sparsity in Dynamics of Spontaneous Subtle Emotions: Analysis and Application

“…LBPTOP, there are other potential feature extraction and representations for micro-expressions based on optical flow, multi-scale wavelet analysis, etc. For instance, Liong et al [12] utilized optical strain, a derivative of optical flow, for the recognition task; building on similar concepts used for expression spotting by Shreve et al [13], [23]. Furthermore, Liu et al [24] encoded statistical information of the main directional optical flows in regions of interests (ROI), which are manually defined with respect to facial landmarks.…”

Effective recognition of facial micro-expressions with video motion magnification

“…Shreve et al [41,42] computed the strain magnitude of optical flow to discriminate micro-expressions from macro-expressions by observing the interval flow in a given threshold. They evaluated their methods on BU [52], USF and USF-HD databases.…”

Study of Facial Micro-expressions in Psychology

“…Very few studies have been conducted on temporal segmentation in face videos. Shreve et al (2009) have proposed a method for temporal segmentation of facial expressions from videos. This is done on the basis of observed facial deformation by calculating facial strain maps and the magnitude of the strain.…”