Towards More Robust Automatic Facial Expression Recognition in Smart Environments

Bernin, Arne; Muller, Larissa; Ghose, Sobin; Luck, Kai von; Grecos, Christos; Wang, Qi; Vogt, Florian

doi:10.1145/3056540.3056546

Cited by 17 publications

(13 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our follow-up study [23] enhanced the experimental setup with the event-based analysis of galvanic skin response combined with facial expressions. For practical guidance, we presented a benchmark [3] of four automatic facial expression analysis systems with three emotion-labelled reference databases and a systematic method for performance analysis and improvement that allows to tailor for specific application needs.…”

Section: Previous Workmentioning

confidence: 99%

“…One standing research challenge is the fully automatic classification of user reactions, for which we present three alternative algorithms as feasible solutions. Our classification methods can be effectively combined with the application-specific clustering approach [3] to increase its robustness for a wide spectrum of user reactions.…”

Section: Previous Workmentioning

confidence: 99%

“…Features are then extracted from the reduced data and machine learning or statistical classification generates the result. Our previous work [3] contains further insights into the nature of FER algorithms.…”

Section: Fer Algorithmsmentioning

confidence: 99%

“…Algorithms may be trained to detect AUs [20] or facial landmarks [21] as an intermediate step or they may be directly trained for facial expression detection on raw input [29]. Their output is typically an independent probability value between 0.0 and 1.0 for every possible expression (see [3] for details).…”

Section: Fer Algorithmsmentioning

confidence: 99%

“…Task-fail means that the participant drifts to the side while jumping or falls off beforehand and does not reach the ski jump at all. We used the Emotient FER algorithm provided as part of the iMotions platform 1 for classifying the video data recorded at 30fps as the algorithm generated good results in our previous benchmarking [3]. We used all 12 provided expression channels, seven basic emotions (joy, anger, surprise, fear, disgust, sadness and contempt) and additionally included: confusion, frustration, neutral emotion, positive emotion and negative emotion.…”

Section: Continuous Eventsmentioning

confidence: 99%

See 4 more Smart Citations

Automatic Classification and Shift Detection of Facial Expressions in Event-Aware Smart Environments

Bernin

Muller

Ghose

et al. 2018

Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference

Self Cite

View full text Add to dashboard Cite

Affective application developers often face a challenge in integrating the output of facial expression recognition (FER) software in interactive systems: although many algorithms have been proposed for FER, integrating the results of these algorithms into applications remains difficult. Due to interand within-subject variations further post-processing is needed. Our work addresses this problem by introducing and comparing three post-processing classification algorithms for FER output applied to an event-based interaction scheme to pinpoint the affective context within a time window. Our comparison is based on earlier published experiments with an interactive cycling simulation in which participants were provoked with game elements and their facial expression responses were analysed by all three algorithms with a human observer as reference. The three post-processing algorithms we investigate are mean fixed-window, matched filter, and Bayesian changepoint detection. In addition, we introduce a novel method for detecting fast transition of facial expressions, which we call emotional shift. The proposed detection pattern is suitable for affective applications especially in smart environments, wherever users' reactions can be tied to events.

show abstract

Section: Previous Workmentioning

confidence: 99%

Section: Previous Workmentioning

confidence: 99%

Section: Fer Algorithmsmentioning

confidence: 99%

Section: Fer Algorithmsmentioning

confidence: 99%

Section: Continuous Eventsmentioning

confidence: 99%

See 3 more Smart Citations

Automatic Classification and Shift Detection of Facial Expressions in Event-Aware Smart Environments

Bernin

Muller

Ghose

et al. 2018

Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference

Self Cite

View full text Add to dashboard Cite

show abstract

Affective Guide for Museum: A System to Suggest Museum Paths Based on Visitors’ Emotions

Altieri

Ceccacci

Giraldi³

et al. 2021

Universal Access in Human-Computer Interaction. Design Methods and User Experience

View full text Add to dashboard Cite

Migratable AI : Investigating Users’ Affect on Identity and Information Migration of a Conversational AI Agent

Tejwani

Katz

Breazeal

2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Conversational AI agents are becoming ubiquitous and provide assistance to us in our everyday activities. In recent years, researchers have explored the migration of these agents across different embodiments in order to maintain the continuity of the task and improve user experience. In this paper, we investigate user's affective responses in different configurations of the migration parameters. We present a 2x2 between-subjects study in a task-based scenario using information migration and identity migration as parameters. We outline the affect processing pipeline from the video footage collected during the study and report user's responses in each condition. Our results show that users reported highest joy and were most surprised when both the information and identity was migrated; and reported most anger when the information was migrated without the identity of their agent.

show abstract

Towards More Robust Automatic Facial Expression Recognition in Smart Environments

Cited by 17 publications

References 29 publications

Automatic Classification and Shift Detection of Facial Expressions in Event-Aware Smart Environments

Automatic Classification and Shift Detection of Facial Expressions in Event-Aware Smart Environments

Affective Guide for Museum: A System to Suggest Museum Paths Based on Visitors’ Emotions

Migratable AI : Investigating Users’ Affect on Identity and Information Migration of a Conversational AI Agent

Contact Info

Product

Resources

About