Action unit detection in 3D facial videos with application in facial expression retrieval and recognition

Abstract

This work introduces a new scheme for action unit detection in 3D facial videos. Sets of features that define action unit activation in a robust manner are proposed. These features are computed based on eight detected facial landmarks on each facial mesh that involve angles, areas and distances. Support vector machine classifiers are then trained using the features of the descriptor in order to perform action unit detection. The proposed AU detection scheme is used in a dynamic 3D facial expression retrieval and recognition pipeline, highlighting the most important AU s, in terms of providing facial expression information, and at the same time, resulting in better performance than the state-of-the-art methodologies.

Modeling and Analysis of Fatigue Detection With Multi-channel Data Fusion

“…muscle activity by electromyogram (EMG) [11] and tissue oxygenation by near infrared spectroscopy (NIRS) [12], etc. In the 1980s, correlations between EEG and cerebral neurological decline have been carried out in foreign Studies showed that EEG is sensitive to fluctuations in alertness, EEG changes more significantly with changes in alertness, and EEG can predict the decline in brain performance caused by sustained mental effort; in the 1990s, EEG research was further advanced and people began to focus on changes in various bands of EEG during cerebral neurasthenia, and studies found that reduced alertness and fatigue in humans The correlation between reduced alertness and fatigue was found to be reflected in different bands of EEG, especially in the theta and alpha waves [13], and the power spectrum of theta and alpha waves increased when people were in a fatigued state; in the 21st century, different bands of EEG were studied in more detail, including theta, alpha and beta bands in EEG and the combination parameters of different bands.…”

The 4D Space-Time Dimensions of Facial Perception

“…While the FACS system has been extensively used with 2D dynamic faces displaying emotional expressions, for example, 2D images (Pantic and Rothkrantz, 2004;Cohn and De la Torre, 2014;Baltrušaitis et al, 2015) and 2D dynamic videos (Donato et al, 1999;Pantic and Rothkrantz, 2004;Bartlett et al, 2006) this study indicates 3D/4D dynamic faces may be more important for low intensity emotions. The FACS system has likewise achieved action unit detection in 3D facial videos or scans (Cosker et al, 2011) for successful emotion detection (Sun et al, 2008;Tsalakanidou and Malassiotis, 2010;Berretti et al, 2013;Reale et al, 2013;Tulyakov et al, 2015;Danelakis et al, 2018); and even more socially related morphologies such as dominance, trustworthiness and attractiveness (Gill et al, 2014) or the influences of ethnicity and gender on how we display emotional expressions differently from one another (Jack et al, 2014;Jandova and Urbanova, 2018).…”

A Standalone Vision Device to Recognize Facial Landmarks and Smile in Real Time Using Raspberry Pi and Sensor

“…A better system in-term of performance and information about facial expression has been proposed [8].A merger based approach has been proposed to recognize multiple facial action units (AU) and estimation of intensity. This technique will combine the multi-task based features and their intensity with Bayesian network based action units that will finally perform the multiple action unit recognition and AU intensity estimation [9].A video based facial emotion recognition system has been proposed that will automatically select the face which is closest to the neutral expression.…”