Facing Driver Frustration: Towards Real-Time In-Vehicle Frustration Estimation Based on Video Streams of the Face

Franz, Oliver; Drewitz, Uwe; Ihme, Klas

doi:10.1007/978-3-030-50732-9_46

Cited by 4 publications

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other side, due to the labels' ambiguity typical of emotional data, for which instead of an objective 'ground truth', a subjective 'gold standard' [33], i.e., an agreed-upon human annotation label, is typically available. Although methods for automatic recognition of frustration from audio-visual cues have been presented, most of these are based on end-to-end models applying cross-entropy loss [9], which despite their excellent convergence speed, present a sub-optimal performance when dealing with ambiguous labels [45]-note that frustration is usually annotated according to a gold standard. Furthermore, frustration can also be confused with other highly aroused emotional states, e. g., anger, or, irritation.…”

Section: Introductionmentioning

confidence: 99%

Supervised Contrastive Learning for Game-Play Frustration Detection from Speech

Song

Parada-Cabaleiro

Liu

et al. 2021

Universal Access in Human-Computer Interaction. Design Methods and User Experience

View full text Add to dashboard Cite

Frustration is a common response during game interactions, typically decreasing a user's engagement and leading to game failure. Artificially intelligent methods capable to automatically detect a user's level of frustration at an early stage are hence of great interest for game designers, since this would enable optimisation of a player's experience in real-time. Nevertheless, research in this context is still in its infancy, mainly relying on the use of pre-trained models and fine-tuning tailored to a specific dataset. Furthermore, this lack in research is due to the limited data available and to the ambiguous labelling of frustration, which leads to outcomes which are not generalisable in the real-world. Meanwhile, contrastive loss has been considered instead of the traditional cross-entropy loss in a variety of machine learning applications, showing to be more robust for system stability alternative in self-supervised learning. Following this trend, we hypothesise that using a supervised contrastive loss might overcome the limitations of the cross-entropy loss yielded by the labels' ambiguity. In fact, our experiments demonstrate that using the supervised contrastive method as a loss function, results improve for the automatic recognition (binary frustration vs nofrustration) of game-induced frustration from speech with an Unweighted Average Recall increase from 86.4 % to 89.9 %.

show abstract

Section: Introductionmentioning

confidence: 99%