Is It Me or the Robot? A Critical Evaluation of Human Affective State Recognition in a Cognitive Task

Abstract

A key goal in human-robot interaction (HRI) is to design scenarios between humanoid robots and humans such that the interaction is perceived as collaborative and natural, yet safe and comfortable for the human. Human skills like verbal and non-verbal communication are essential elements as humans tend to attribute social behaviors to robots. However, aspects like the uncanny valley and different technical affinity levels can impede the success of HRI scenarios, which has consequences on the establishment of long-term interaction qualities like trust and rapport. In the present study, we investigate the impact of a humanoid robot on human emotional responses during the performance of a cognitively demanding task. We set up three different conditions for the robot with increasing levels of social cue expressions in a between-group study design. For the analysis of emotions, we consider the eye gaze behavior, arousal-valence for affective states, and the detection of action units. Our analysis reveals that the participants display a high tendency toward positive emotions in presence of a robot with clear social skills compared to other conditions, where we show how emotions occur only at task onset. Our study also shows how different expression levels influence the analysis of the robots' role in HRI. Finally, we critically discuss the current trend of automatized emotion or affective state recognition in HRI and demonstrate issues that have direct consequences on the interpretation and, therefore, claims about human emotions in HRI studies.

Keeping social distance in a classroom while interacting via a telepresence robot: a pilot study

“…Lastly, forthcoming research endeavors should comprehensively investigate non-verbal language cues. In human communication, the significance of gaze and leaning is evident right from the initial contact moment, a fact validated also in human-robot interaction by Kanda et al ( 2008 ) and Jirak et al ( 2022 ). In these situations, of course, the researchers need to acknowledge the technical limitations of TPRs, such as relatively rudimental body or limited eye-sight, as highlighted by Talisainen et al ( 2023 ).…”

I am Only Happy When There is Light: The Impact of Environmental Changes on Affective Facial Expressions Recognition

“…As a baseline, the participants performed the task without the presence of the robot. The analysis of the "social" condition revealed mostly positive arousal-valence patterns elicited by the participants compared to the mixed results in the "non-social" condition or mostly neutral states in the baseline [13]. Hence, we consider the "social" subset as reasonable data baseline.…”