Spontaneous and posed smile recognition based on spatial and temporal patterns of facial EMG

Perusquía-Hernández, Monica; Hirokawa, Masakazu; Suzuki, Kenji

doi:10.1109/acii.2017.8273651

Cited by 14 publications

(8 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Their results showed that the use of PCA prior to classification improved the classifier's accuracy, achieving an average classification rate of 69.5% across the six basic emotions (anger, disgust, fear, happiness, sadness and surprise). Perusquia-Hernandez et al [47] investigated the recognition of spontaneous vs. posed smiles, using spatial and temporal patterns of facial EMG. Due to the unbalanced nature of the collected data, they undersampled the majority class to match the minority class samples (as in [55]).…”

Section: Emotion Sensing From Facial Emgmentioning

confidence: 99%

“…Therefore, going forward, the presented results are based on the DWT-Haar approximation of the EMG signal due to its efficient computation. A significant number of studies, in the domain of facial EMG classification, have shown temporal features to be the most informative [25,30,47,57]. Based on these findings, the time and time-frequency domain features were extracted from the DWT approximation of the signal.…”

Section: Feature Extractionmentioning

confidence: 99%

“…This allows emotion recognition to be framed as a 4-class classification problem. Three classifiers, which have been utilised to varying degrees of success in existing facial EMG literature [25,30,47,57], SVMs (with Gaussian kernel); kNN (with various values for k); and LDA were evaluated using the (subject-independent) leave-one-subject-out (LOSO) cross-validation strategy. The best classification results for each classifier can be seen in Fig.…”

Section: Affect Classificationmentioning

confidence: 99%

See 2 more Smart Citations

Facial Electromyography-based Adaptive Virtual Reality Gaming for Cognitive Training

Reidy

Chan

Nduka

et al. 2020

Proceedings of the 2020 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Cognitive training has shown promising results for delivering improvements in human cognition related to attention, problem solving, reading comprehension and information retrieval. However, two frequently cited problems in cognitive training literature are a lack of user engagement with the training programme, and a failure of developed skills to generalise to daily life. This paper introduces a new cognitive training (CT) paradigm designed to address these two limitations by combining the benefits of gamification, virtual reality (VR), and affective adaptation in the development of an engaging, ecologically valid, CT task. Additionally, it incorporates facial electromyography (EMG) as a means of determining user affect while engaged in the CT task. This information is then utilised to dynamically adjust the game's difficulty in real-time as users play, with the aim of leading them into a state of flow. Affect recognition rates of 64.1% and 76.2%, for valence and arousal respectively, were achieved by classifying a DWT-Haar approximation of the input signal using kNN. The affect-aware VR cognitive training intervention was then evaluated with a control group of older adults. The results obtained substantiate the notion that adaptation techniques can lead to greater feelings of competence and a more appropriate challenge of the user's skills. CCS CONCEPTS • Human-centered computing → Interactive systems and tools; • Computing methodologies → Virtual reality; Machine learning.

show abstract

Section: Emotion Sensing From Facial Emgmentioning

confidence: 99%

Section: Feature Extractionmentioning

confidence: 99%

Section: Affect Classificationmentioning

confidence: 99%

See 1 more Smart Citation

Facial Electromyography-based Adaptive Virtual Reality Gaming for Cognitive Training

Reidy

Chan

Nduka

et al. 2020

Proceedings of the 2020 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

show abstract

“…Many smiles peak multiple times without fully returning to the baseline. This has led to several peak detection approaches described previously (Mavadati et al, 2016;Perusquía-Hernández et al, 2017a). Moreover, as a short smile composed from the first bout of laughter transforms in sustained laughter, one spatially identifiable smile unit is composed of many temporally-defined smile units which are more challenging to identify from a single feature type (Perusquía-Hernández et al, 2017b).…”

Section: The Shape Of a Smilementioning

confidence: 99%

Are people happy when they smile? Affective assessments based on automatic smile genuineness identification

Perusquía-Hernández¹

2020

Preprint

View full text Add to dashboard Cite

Smiles are one of the most ubiquitous facial expressions. They are often interpreted as a signalling cue of positive emotion. However, as any other facial expression, smiles can also be voluntarily fabricated, masked or inhibited with different communication goals. This review discusses automatic identification of smile genuineness. First, emotions and their bodily manifestation are introduced. Second, an overview of the literature on different types of smiles is provided. Afterwards, different techniques used to investigate smile production are described. These techniques range from human video-coding, bio-signal inspection, and novel sensors that, together with automated techniques using machine learning, aim to investigate facial expression characteristic’s beyond human perception. Next, a general summary of the spatio-temporal shape of a smile is provided. Finally, the remaining challenges regarding individual and cultural differences are discussed.

show abstract

“…Physiological signals have shown tremendous results in detecting different emotions among people. Perusquia-Hernandez et al [12] was able to differentiate between spontaneous and posed smiles using features from facial EMG.…”

confidence: 99%

Are paired or single stimuli better to recognize genuine and posed smiles from observers' GSR

Rahman¹,

Hossain²

2020

Preprint

View full text Add to dashboard Cite

<div>Smile recognition plays a vital role in human-human and human-computer interactions. This paper demonstrates a system to recognize the genuine and posed smiles by sensing observers’ galvanic skin response (GSR), while watching sets of images and videos. The smiles were shown either in ‘paired’ or in ‘single’ forms. Here, ‘paired’ means that the same smiler was seen in both genuine and posed</div><div>smile forms, otherwise the condition is referred to as ‘single’. The GSR signals were recorded and processed, and several time-domain and frequency-domain features were extracted from the processed GSR signals. Classification accuracies were found to be as high as 93.6% and 91.4% from paired and single conditions respectively. In comparison, observers were verbally 59.8% and 56.2% correct. Our results demonstrate that human subconscious responses (i.e. GSR signals) is better than their own verbal response, where the paired condition is slightly better than the single condition.</div>

show abstract

Spontaneous and posed smile recognition based on spatial and temporal patterns of facial EMG

Cited by 14 publications

References 14 publications

Facial Electromyography-based Adaptive Virtual Reality Gaming for Cognitive Training

Facial Electromyography-based Adaptive Virtual Reality Gaming for Cognitive Training

Are people happy when they smile? Affective assessments based on automatic smile genuineness identification

Are paired or single stimuli better to recognize genuine and posed smiles from observers' GSR

Contact Info

Product

Resources

About