Affective Robot Story-Telling Human-Robot Interaction: Exploratory Real-Time Emotion Estimation Analysis Using Facial Expressions and Physiological Signals

Abstract

Affective human-robot interaction is still an active area of research in part due to the great advances in artificial intelligence. Now, the design of autonomous devices that work in real therapeutic environments has become a plausible reality. Affective human-robot interaction requires a robot to analyze the emotional state of the human interlocutor and interpret emotional responses that can be used, not merely in the interaction but, for example, to provoke desired therapeutic responses. It is, therefore, necessary to broaden experimental techniques into more realistic paradigms, where the capacity of emotion estimation can be completely explored. This exploratory paper proposes a realistic experimental paradigm in which the robot employs a dramatic story to evoke emotions in the users, and tests previously self-designed methodologies to be able to make estimates of the users' emotional state in real-time. Regardless of the multiple impediments and restrictions, and all the aspects that could still be improved, this paper can outline the feasibility of the proposed methodology in realistic scenarios.

Responsible Human-Robot Interaction with Anthropomorphic Service Robots: State of the Art of an Interdisciplinary Research Challenge

“…Observations and analytical sensors can be used to measure the emotions of an individual using audio [115], physiological [116] visual signals [3,117]. A literature review on the use of physiological signals in HRI to detect emotions points out that the baseline of the physiological signals should be gathered before recognizing the influence of the emotions [118,119]. [120] indicates that the first encounter with a robot should take place before the experiment starts.…”

DRER: Deep Learning–Based Driver’s Real Emotion Recognizer

“…Owing to the benefits of flexibility and practicality, studies are actively conducted in areas that require real-time emotion recognition in various situations. Val-Calvo et al [ 26 ] proposed self-designed methodologies to estimate users’ emotional state in real time. They used EDA, blood volume pressure (BVP) and EEG to analyze the statistical correlation between experienced emotions and the properties of the set of features.…”

“…The most commonly used bio-physiology signals are electroencephalogram (EEG), electrocardiogram (ECG), photoplethysmography (PPG) and electrodermal activity (EDA). Moreover, some studies using both facial expressions and bio-physiology signals achieved high accuracy in the case of emotion recognition [ 24 , 25 ] and performed well in recognizing various emotion classes [ 26 , 27 ]. All these studies are based on deep learning algorithms.…”

Emotion Recognition for Human-Robot Interaction: Recent Advances and Future Perspectives

“…In Val-Calvo et al ( 2020 ), an interesting analysis of the possibilities of ER in HRI was performed by using facial images, EEG, GSR, and blood pressure. In a realistic HRI scenario, a Pepper robot dynamically drives subjects' emotional responses by story-telling and multimedia stimuli.…”