Smile Action Unit detection from distal wearable Electromyography and Computer Vision

Abstract

Distal facial Electromyography (EMG) can be used to detect smiles and frowns with reasonable accuracy. It capitalizes on volume conduction to detect relevant muscle activity, even when the electrodes are not placed directly on the source muscle. The main advantage of this method is to prevent occlusion and obstruction of the facial expression production, whilst allowing EMG measurements. However, measuring EMG distally entails that the exact source of the facial movement is unknown. We propose a novel method to estimate specific Facial Action Units (AUs) from distal facial EMG and Computer Vision (CV). This method is based on Independent Component Analysis (ICA), Non-Negative Matrix Factorization (NNMF), and sorting of the resulting components to determine which is the most likely to correspond to each CV-labeled action unit (AU). Performance on the detection of AU06 (Orbicularis Oculi) and AU12 (Zygomaticus Major) was estimated by calculating the agreement with Human Coders. The results of our proposed algorithm showed an accuracy of 81% and a Cohen's Kappa of 0.49 for AU6; and accuracy of 82% and a Cohen's Kappa of 0.53 for AU12. This demonstrates the potential of distal EMG to detect individual facial movements. Using this multimodal method, several AU synergies were identified. We quantified the co-occurrence and timing of AU6 and AU12 in posed and spontaneous smiles using the human-coded labels, and for comparison, using the continuous CV-labels. The co-occurrence analysis was also performed on the EMG-based labels to uncover the relationship between muscle synergies and the kinematics of visible facial movement.

The spatio-temporal features of perceived-as-genuine and deliberate expressions

“…To reduce the dimensionality and extract the low-dimensional features, a nonnegative matrix factorization was applied to the time-series data of the AUs [44][45][46]. This approach helps obtain interpretable features in a low-dimensional space [44].…”

“…It should be noted that the assessment of facial movements is largely dependent on the target stimuli and their nature [84], but the state-of-the-art AU detection system comparisons provided average F1 scores of .56-.59 [85]. Perusquia-Herna ´ndez et al [46] also indicate the existence of entanglement between upper lip raising (AU10) and lip corner pulling (AU12). Replication studies with a more sophisticated facial movement detection system are awaited.…”

I'm Transforming! Effects of Visual Transitions to Change of Avatar on the Sense of Embodiment in AR

“…This allowed us to evaluate the user's physical performance across avatars and conditions. After the experiment, the acquired sEMG values were band-pass filtered from 25 to 200 Hz and the root means square (RMS) was calculated using a sliding window of 100 milliseconds following the methodology used in the study of Perusquía-Hernández et al (2021) [44].…”

Smartphone User Identification/Authentication Using Accelerometer and Gyroscope Data

“…The IBA metric combines a measure of how dominating the class with the highest individual Accuracy rate is with an unbiased index of its Overall Accuracy [45]. Previous studies indicated that all performance metrics, with the exception of the AUC, were weakened by skewed data distributions [61][62][63]. For these reasons, the AUC in addition to the aforementioned metrics were selected to gain a better insight into the performance of the proposed identification system using a specific classifier.…”