Engagement Perception and Generation for Social Robots and Virtual Agents

Corrigan, Lee J.; Peters, Christopher; Küster, Dennis; Castellano, Ginevra

doi:10.1007/978-3-319-31056-5_4

Cited by 26 publications

(23 citation statements)

References 23 publications

(32 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Social engagement very often also includes an affective component (see for example Corrigan et al, 2016 ; Biancardi et al, 2019a ; Sohail et al, 2019 ; Youssef et al, 2019 ). The definition of the affective component of engagement often remains vague.…”

Section: Definition Of Engagementmentioning

confidence: 99%

Engagement in Human-Agent Interaction: An Overview

et al. 2020

Self Cite

View full text Add to dashboard Cite

Engagement is a concept of the utmost importance in human-computer interaction, not only for informing the design and implementation of interfaces, but also for enabling more sophisticated interfaces capable of adapting to users. While the notion of engagement is actively being studied in a diverse set of domains, the term has been used to refer to a number of related, but different concepts. In fact it has been referred to across different disciplines under different names and with different connotations in mind. Therefore, it can be quite difficult to understand what the meaning of engagement is and how one study relates to another one accordingly. Engagement has been studied not only in human-human, but also in human-agent interactions i.e., interactions with physical robots and embodied virtual agents. In this overview article we focus on different factors involved in engagement studies, distinguishing especially between those studies that address task and social engagement, involve children and adults, are conducted in a lab or aimed for long term interaction. We also present models for detecting engagement and for generating multimodal behaviors to show engagement.

show abstract

Section: Definition Of Engagementmentioning

confidence: 99%

Engagement in Human-Agent Interaction: An Overview

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…In particular, we focus on the user's level of engagement. So, we now present our computational model of user's engagement based on the works of Corrigan et al ( 2016 ) and Sidner and Dzikovska ( 2005 ). In our model, user's engagement can be expressed at three different levels, corresponding to different types of non-verbal signals:…”

Section: System Architecturementioning

confidence: 99%

“…Cognitive engagement –(Corrigan et al, 2016 ) claims that “frowning may indicate effortful processing suggesting high levels of cognitive engagement.” The same work also refers to signals such as “looking for a brief interval outside the scene” as indicators of cognitive engagement.…”

Section: System Architecturementioning

confidence: 99%

“…A detailed summary of engagement definitions in human-agent interaction is provided in Glas and Pelachaud ( 2015a ). Among others, it can be defined as “the value that a participant in an interaction attributes to the goal of being together with the other participant(s) and of continuing the interaction” (Poggi, 2007 ), and as “the process by which participants involved in an interaction start, maintain and terminate an interaction” (Sidner and Dzikovska, 2005 ; Corrigan et al, 2016 ).…”

Section: Introductionmentioning

confidence: 99%

“…Cognitive engagement-(Corrigan et al, 2016) claims that "frowning may indicate effortful processing suggesting high levels of cognitive engagement." The same work also refers to signals such as "looking for a brief interval outside the scene" as indicators of cognitive engagement.• Affective engagement-Smiling could indicate that a person is enjoying the interaction, while some postures (e.g., crossed arms, hands in pockets) or posture shifts can indicate a lack of engagement.The Affective and Cognitive Engagement Detection module is based on a Long Short-Term Memory (LSTM) prediction model using Recurrent Neural Networks implemented with the Keras toolkit and TensorFlow.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Managing an Agent's Self-Presentational Strategies During an Interaction

Biancardi¹,

Mancini

Lerner

et al. 2019

Front. Robot. AI

View full text Add to dashboard Cite

In this paper we present a computational model for managing the impressions of warmth and competence (the two fundamental dimensions of social cognition) of an Embodied Conversational Agent (ECA) while interacting with a human. The ECA can choose among four different self-presentational strategies eliciting different impressions of warmth and/or competence in the user, through its verbal and non-verbal behavior. The choice of the non-verbal behaviors displayed by the ECA relies on our previous studies. In our first study, we annotated videos of human-human natural interactions of an expert on a given topic talking to a novice, in order to find associations between the warmth and competence elicited by the expert's non-verbal behaviors (such as type of gestures, arms rest poses, smiling). In a second study, we investigated whether the most relevant non-verbal cues found in the previous study were perceived in the same way when displayed by an ECA. The computational learning model presented in this paper aims to learn in real-time the best strategy (i.e., the degree of warmth and/or competence to display) for the ECA, that is, the one which maximizes user's engagement during the interaction. We also present an evaluation study, aiming to investigate our model in a real context. In the experimental scenario, the ECA plays the role of a museum guide introducing an exposition about video games. We collected data from 75 visitors of a science museum. The ECA was displayed in human dimension on a big screen in front of the participant, with a Kinect on the top. During the interaction, the ECA could adopt one of 4 self-presentational strategies during the whole interaction, or it could select one strategy randomly for each speaking turn, or it could use a reinforcement learning algorithm to choose the strategy having the highest reward (i.e., user's engagement) after each speaking turn.

show abstract

Engagement estimation of the elderly from wild multiparty human–robot interaction

Zhang

Zheng

Thalmann

2022

Computer Animation & Virtual

View full text Add to dashboard Cite

The use of social robots in healthcare systems or nursing homes to assist the elderly and their caregivers will be becoming common, where robots' understanding of engagement of the elderly is important. Traditional engagement estimation (EE) often requires expert involvement in a controlled dyadic interaction environment. In this article, we propose a supervised machine learning method to estimate the engagement state of the elderly in a multiparty human-robot interaction (HRI) scenario from the real-world video recording as input. The method is built upon the basic concept of engagement in geriatric psychiatry and HRI video representations. It adapts pretrained models to extract behavior, affective, and visual signals to form the multi-modal features. These features are then fed into a neural network made of a self-attention mechanism and average pooling for individual learning, a graph attention network for group learning and a fully connected layer to estimate the engagement. We tested the proposed method using 43 wild multiparty elderly robot interaction (ERI) videos. The experimental results show that our method is capable of detecting the key participants and estimating the engagement state of the elderly effectively. Also our study demonstrates the signals from side-participants in the main interaction group considerably contribute to the EE of the elderly in the multiparty ERI.

show abstract

Engagement Perception and Generation for Social Robots and Virtual Agents

Cited by 26 publications

References 23 publications

Engagement in Human-Agent Interaction: An Overview

Engagement in Human-Agent Interaction: An Overview

Managing an Agent's Self-Presentational Strategies During an Interaction

Engagement estimation of the elderly from wild multiparty human–robot interaction

Contact Info

Product

Resources

About