The Invisible Potential of Facial Electromyography

Perusquía-Hernández, Monica; Ayabe‐Kanamura, Saho; Suzuki, Kenji; Kumano, Shiro

doi:10.1145/3290605.3300379

Cited by 23 publications

(8 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The wearable community has explored facial expression monitoring with different sensing modalities embedded in accessories. The various sensing methods include cameras [16,35], IMU [31,20], light [22], capacitive [39], piezoelectric [40], electromyography (EMG) [41], mechanomyography [28,2], audio based solutions [2,33,42], and many others. Table 1 shows a comparison of the state-of-the-art approaches.…”

Section: Facial Monitoring With Wearablesmentioning

confidence: 99%

InMyFace: Inertial and Mechanomyography-Based Sensor Fusion for Wearable Facial Activity Recognition

Bello¹,

Marin²,

Suh³

et al. 2023

Preprint

View full text Add to dashboard Cite

Recognizing facial activity is a well-understood (but non-trivial) computer vision problem. However, reliable solutions require a camera with a good view of the face, which is often unavailable in wearable settings. Furthermore, in wearable applications, where systems accompany users throughout their daily activities, a permanently running camera can be problematic for privacy (and legal) reasons. This work presents an alternative solution based on the fusion of wearable inertial sensors, planar pressure sensors, and acoustic mechanomyography (muscle sounds). The sensors were placed unobtrusively in a sports cap to monitor facial muscle activities related to facial expressions. We present our integrated wearable sensor system, describe data fusion and analysis methods, and evaluate the system in an experiment with thirteen subjects from different cultural backgrounds (eight countries) and both sexes (six women and seven men). In a one-model-per-user scheme and using a late fusion approach, the system yielded an average F1 score of 85.00% for the case where all sensing modalities are combined. With a cross-user validation and a one-model-for-all-user scheme, an F1 score of 79.00% was obtained for thirteen participants (six females and seven males). Moreover, in a hybrid fusion (cross-user) approach and six classes, an average F1 score of 82.00% was obtained for eight users. The results are competitive with state-of-the-art non-camera-based solutions for a cross-user study. In addition, our unique set of participants demonstrates the inclusiveness and generalizability of the approach.

show abstract

Section: Facial Monitoring With Wearablesmentioning

confidence: 99%

InMyFace: Inertial and Mechanomyography-Based Sensor Fusion for Wearable Facial Activity Recognition

Bello¹,

Marin²,

Suh³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Despite the many benefits, several pitfalls are expected in using visual data to analyze facial expressions: First, it is expected that visual data and CV will be more reliable in identifying activation associated with visible body or skin movement. In scenarios involving isometric muscle activation, (muscular activation without apparent bodily movement) CV validity may be dubious [ 16 ]. Another major challenge in analyzing muscle activation with CV is in the identification of the movement source.…”

Section: Introductionmentioning

confidence: 99%

Similarities and disparities between visual analysis and high-resolution electromyography of facial expressions

et al. 2022

View full text Add to dashboard Cite

Computer vision (CV) is widely used in the investigation of facial expressions. Applications range from psychological evaluation to neurology, to name just two examples. CV for identifying facial expressions may suffer from several shortcomings: CV provides indirect information about muscle activation, it is insensitive to activations that do not involve visible deformations, such as jaw clenching. Moreover, it relies on high-resolution and unobstructed visuals. High density surface electromyography (sEMG) recordings with soft electrode array is an alternative approach which provides direct information about muscle activation, even from freely behaving humans. In this investigation, we compare CV and sEMG analysis of facial muscle activation. We used independent component analysis (ICA) and multiple linear regression (MLR) to quantify the similarity and disparity between the two approaches for posed muscle activations. The comparison reveals similarity in event detection, but discrepancies and inconsistencies in source identification. Specifically, the correspondence between sEMG and action unit (AU)-based analyses, the most widely used basis of CV muscle activation prediction, appears to vary between participants and sessions. We also show a comparison between AU and sEMG data of spontaneous smiles, highlighting the differences between the two approaches. The data presented in this paper suggests that the use of AU-based analysis should consider its limited ability to reliably compare between different sessions and individuals and highlight the advantages of high-resolution sEMG for facial expression analysis.

show abstract

“…Other experimental methods to elicit natural emotional responses include manipulations to elicit spontaneous versus posed smiles [10], such as asking users to watch funny videos and asking them to withhold from laughing, thus naturally eliciting genuine smiles. Such subtle microexpressions can be measured through facial distal electromyography (EMG) [25].…”

Section: Introductionmentioning

confidence: 99%

MEEC: First Workshop on Momentary Emotion Elicitation and Capture

Ali

Perusquía-Hernández

Denman

et al. 2020

Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

Recognizing human emotions and responding appropriately has the potential to radically change the way we interact with technology. However, to train machines to sensibly detect and recognize human emotions, we need valid emotion ground truths. A fundamental challenge here is the momentary emotion elicitation and capture (MEEC) from individuals continuously and in real-time, without adversely affecting user experience. In this first edition of the one-day CHI 2020 workshop, we will (a) explore and define novel elicitation tasks (b) survey sensing and annotation techniques (c) create a taxonomy of when and where to apply an elicitation method.

show abstract

The Invisible Potential of Facial Electromyography

Cited by 23 publications

References 37 publications

InMyFace: Inertial and Mechanomyography-Based Sensor Fusion for Wearable Facial Activity Recognition

InMyFace: Inertial and Mechanomyography-Based Sensor Fusion for Wearable Facial Activity Recognition

Similarities and disparities between visual analysis and high-resolution electromyography of facial expressions

MEEC: First Workshop on Momentary Emotion Elicitation and Capture

Contact Info

Product

Resources

About