An Autonomous Cognitive Empathy Model Responsive to Users’ Facial Emotion Expressions

Abstract

Successful social robot services depend on how robots can interact with users. The effective service can be obtained through smooth, engaged, and humanoid interactions in which robots react properly to a user’s affective state. This article proposes a novel Automatic Cognitive Empathy Model, ACEM, for humanoid robots to achieve longer and more engaged human-robot interactions (HRI) by considering humans’ emotions and replying to them appropriately. The proposed model continuously detects the affective states of a user based on facial expressions and generates desired, either parallel or reactive, empathic behaviors that are already adapted to the user’s personality. Users’ affective states are detected using a stacked autoencoder network that is trained and tested on the RAVDESS dataset. The overall proposed empathic model is verified throughout an experiment, where different emotions are triggered in participants and then empathic behaviors are applied based on proposed hypothesis. The results confirm the effectiveness of the proposed model in terms of related social and friendship concepts that participants perceived during interaction with the robot.

Affective Communication for Socially Assistive Robots (SARs) for Children with Autism Spectrum Disorder: A Systematic Review

“…The robot behavior is then adapted according to the recognized emotions of the person. Through behavior, emotions can be expressed, and decisions can be made according to the context, as shown in [ 63 ], where the authors state that empathic behavior has two levels of empathic responses: parallel and reactive. Parallel empathy describes empathic responses that mimic the target’s emotions, whereas reactive empathy describes empathic responses that foster objective verbal and non-verbal actions to reduce the target’s distress.…”

“…To achieve more extended interaction, the Automatic Cognitive Empathy Model (ACEM) was proposed for humanoid robots. Bagheri et al [ 63 ] considered the definition of empathy proposed by Davis as “a set of constructs that connects the responses of one individual to the experience of another” [ 83 ]. They considered two kinds of empathy: cognitive and affective.…”

“…Other studies have integrated personality, such as TAME [ 66 , 111 , 112 ], iGrace [ 91 ], EMIA [ 101 ], and ECAs [ 63 , 102 ]. These models have used the Big Five model [ 48 ] and OCEAN, representing the following five dimensions: openness, conscientiousness, extraversion, agreeableness, neuroticism.…”

“…These models have used the Big Five model [ 48 ] and OCEAN, representing the following five dimensions: openness, conscientiousness, extraversion, agreeableness, neuroticism. The authors of [ 63 ] incorporated two types of personality into the robot, extrovert and introvert.…”

Expect the Unexpected: Leveraging the Human-Robot Ecosystem to Handle Unexpected Robot Failures

“…2021) Develop mechanisms for robots to understand and react to emotions, like pain, fear, panic, confusion, and distress, to correctly identify warning conditions requiring a reaction. Kulic and Croft (2007), Leite et al (2013), Le andLee (2014), Jung (2017), Bagheri et al (2020), de Kervenoael et al (2020, Spezialetti et al (2020), Weidemann and Rußwinkel (2021) Perform task analysis, communication analysis, workflow analysis, and usability studies of robot interactions with people in the HRE (users, bystanders, customer service agents, etc.). Test in real-world environments, under normal and degraded conditions, to identify challenges, potential degraded states, and conflicts originating from interactions within the HRE.…”