Facilitating the Child–Robot Interaction by Endowing the Robot with the Capability of Understanding the Child Engagement: The Case of Mio Amico Robot

Filippini, Chiara; Spadolini, Edoardo; Cardone, Daniela; Bianchi, Domenico; Preziuso, Maurizio; Sciarretta, Christian; Cimmuto, Valentina del; Lisciani, Davide

doi:10.1007/s12369-020-00661-w

Cited by 29 publications

(27 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Learning the users' emotions is essential in many applications, such as social robots used as communication assistance for education and entertainment [9] This article describes an acceptable and natural interaction between Social Robots and children. The thermal facial reaction of youngsters, i.e., the nose tip temperature signal, was recorded and classi ed in realtime using the Mio Amico Robot during an experimental session.…”

Section: Related Workmentioning

confidence: 99%

“…Two typical categorical representations for emotions exist Ekman representation [2] which includes anger, happiness, disgust, surprise, fear, and sadness, and Plutchik model [3], [4], which includes Ekman's six emotions in addition to two labels: trust and anticipation. Emotion recognition systems from facial expressions and images have been widely used [6], [7], [8], and [9]. Hand gesture recognition is used as a part of Human action recognition (HAR) [10]).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning for emotion analysis in Arabic tweets

Khalil

Houby

Mohamed

2021

Preprint

View full text Add to dashboard Cite

Expressing our emotions using text and emojis expressions became widespread through social media such as Facebook, Instagram, Twitter, Weibo, and LinkedIn. Nowadays, both organizations and individuals are interested in using social media to analyze people's opinions and extract sentiments and emotions. We proposed a model for multilabel emotion classification, using a bidirectional Long Short-term Memory BiLSTM deep network. It is evaluated on the Arabic tweets' dataset provided by SemEval 2018 for the E-c task. Several preprocessing steps, including ARLSTEM with some modifications, replacing emojis with corresponding text meaning from a manually built lexicon, and feature vector representation using Aravec word embedding is applied. The novelty in our research that it examines the effect of hyperparameter tuning on model performance, and it uses BiLSTM in all of its deep neural network layers. The proposed model achieves a comparable performance with state-of-the-art models using different machine learning and deep learning techniques. The system achieves about 9% enhancement in validation accuracy compared with the last best model in the same task using Support Vector classifier SVC; it outperforms the other deep neural networks (UNCCTeam) based on fully connected layers in micro F1 metric of about 4.4%.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Deep Learning for emotion analysis in Arabic tweets

Khalil

Houby

Mohamed

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, studies of robots used in autism therapy have demonstrated that robots can influence the way children interact with others [ 1 ], just as many previous works have shown that employing robots in the rehabilitation field has considerable effects on patient improvement [ 7 ]. In all these applications, robots are usually designed to interact with humans in a spontaneous, interpersonal way, often with specific social–emotional goals [ 8 ]. However, these agents are still struggling to interact with users in a human-like way, which is an issue that has proved to be challenging to overcome.…”

Section: Introductionmentioning

confidence: 99%

“…The data collected during the experiment were also analyzed with the developed CNN-based FER model. To compute the accuracy of both models (the CNN-based FER module and the NAO's FER module), the data recorded during the experimental sessions were also analyzed with the FER software, i.e., FaceReader 7 (developed by VicarVision and Noldus Information Technology, https://www.noldus.com/human-behavior-research/products/facereader accessed on 25 September 2021) already used in the literature [8] and validated in various research studies [23]. Such software provided the ground truth data on FER and arousal detection.…”

Section: Introductionmentioning

confidence: 99%

Improving Human–Robot Interaction by Enhancing NAO Robot Awareness of Human Facial Expression

Filippini

Perpetuini

Cardone

2021

Sensors

Self Cite

View full text Add to dashboard Cite

An intriguing challenge in the human–robot interaction field is the prospect of endowing robots with emotional intelligence to make the interaction more genuine, intuitive, and natural. A crucial aspect in achieving this goal is the robot’s capability to infer and interpret human emotions. Thanks to its design and open programming platform, the NAO humanoid robot is one of the most widely used agents for human interaction. As with person-to-person communication, facial expressions are the privileged channel for recognizing the interlocutor’s emotional expressions. Although NAO is equipped with a facial expression recognition module, specific use cases may require additional features and affective computing capabilities that are not currently available. This study proposes a highly accurate convolutional-neural-network-based facial expression recognition model that is able to further enhance the NAO robot’ awareness of human facial expressions and provide the robot with an interlocutor’s arousal level detection capability. Indeed, the model tested during human–robot interactions was 91% and 90% accurate in recognizing happy and sad facial expressions, respectively; 75% accurate in recognizing surprised and scared expressions; and less accurate in recognizing neutral and angry expressions. Finally, the model was successfully integrated into the NAO SDK, thus allowing for high-performing facial expression classification with an inference time of 0.34 ± 0.04 s.

show abstract

“…Numerous educational scenarios with different learning subjects have been implemented, such as learning handwriting [13,14], a second language [15], and social emotional learning [16]. Additionally, many CRI scenarios have been designed for exploring special research goals, for example, understanding child engagement [17], defining parameters in a long-term interaction [12], and finding appropriate ways to adjust the curriculum on CRI [18]. This extensive range of applications and research results indicate the great potentials of robotic use in children's edification.…”

Section: Introductionmentioning

confidence: 99%

Visual Robotic Perception System with Incremental Learning for Child–Robot Interaction Scenarios

et al. 2021

View full text Add to dashboard Cite

This paper proposes a novel lightweight visual perception system with Incremental Learning (IL), tailored to child–robot interaction scenarios. Specifically, this encompasses both an action and emotion recognition module, with the former wrapped around an IL system, allowing novel actions to be easily added. This IL system enables the tutor aspiring to use robotic agents in interaction scenarios to further customize the system according to children’s needs. We perform extensive evaluations of the developed modules, achieving state-of-the-art results on both the children’s action BabyRobot dataset and the children’s emotion EmoReact dataset. Finally, we demonstrate the robustness and effectiveness of the IL system for action recognition by conducting a thorough experimental analysis for various conditions and parameters.

show abstract

Facilitating the Child–Robot Interaction by Endowing the Robot with the Capability of Understanding the Child Engagement: The Case of Mio Amico Robot

Cited by 29 publications

References 50 publications

Deep Learning for emotion analysis in Arabic tweets

Deep Learning for emotion analysis in Arabic tweets

Improving Human–Robot Interaction by Enhancing NAO Robot Awareness of Human Facial Expression

Visual Robotic Perception System with Incremental Learning for Child–Robot Interaction Scenarios

Contact Info

Product

Resources

About