Analysis of Engagement and User Experience with a Laughter Responsive Social Robot

Turker, Bekir Berker; Buçinca, Zana; Erzin, Engin; Yemez, Y.; Sezgin, Metin

doi:10.21437/interspeech.2017-1395

Cited by 20 publications

(12 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…facial movement, expression, head pose) [34,36], conversational behaviors (e.g. voice activity, adjacency pair, backchannel, turn length) [18,35,37], laughing [38], and posture [39]. Engagement recognition modules based on the multi-modal features were implemented in agent systems and empirically tested with real users [36].…”

Section: B) Engagement Recognitionmentioning

confidence: 99%

Engagement recognition by a latent character model based on multimodal listener behaviors in spoken dialogue

Inoue

Lala

Takanashi

et al. 2018

SIP

View full text Add to dashboard Cite

Engagement represents how much a user is interested in and willing to continue the current dialogue. Engagement recognition will provide an important clue for dialogue systems to generate adaptive behaviors for the user. This paper addresses engagement recognition based on multimodal listener behaviors of backchannels, laughing, head nodding, and eye gaze. In the annotation of engagement, the ground-truth data often differs from one annotator to another due to the subjectivity of the perception of engagement. To deal with this, we assume that each annotator has a latent character that affects his/her perception of engagement. We propose a hierarchical Bayesian model that estimates both engagement and the character of each annotator as latent variables. Furthermore, we integrate the engagement recognition model with automatic detection of the listener behaviors to realize online engagement recognition. Experimental results show that the proposed model improves recognition accuracy compared with other methods which do not consider the character such as majority voting. We also achieve online engagement recognition without degrading accuracy.

show abstract

Section: B) Engagement Recognitionmentioning

confidence: 99%

Engagement recognition by a latent character model based on multimodal listener behaviors in spoken dialogue

Inoue

Lala

Takanashi

et al. 2018

SIP

View full text Add to dashboard Cite

show abstract

“…We take the engagement level of WoZ setup experiments as a gold standard in JOKER dataset. We evaluate the autonomous setup experiments by comparing with the gold standard engagement measurements as in [17].…”

Section: B Engagement Measuresmentioning

confidence: 99%

Multifaceted Engagement in Social Interaction with a Machine: The JOKER Project

Devillers

Rosset

Duplessis

et al. 2018

2018 13th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2018)

Self Cite

View full text Add to dashboard Cite

This paper addresses the problem of evaluating engagement of the human participant by combining verbal and nonverbal behaviour along with contextual information. This study will be carried out through four different corpora. Four different systems designed to explore essential and complementary aspects of the JOKER system in terms of paralinguistic/linguistic inputs were used for the data collection. An annotation scheme dedicated to the labeling of verbal and non-verbal behavior have been designed. From our experiment, engagement in HRI should be multifaceted.

show abstract

“…summarizes the issues regarding engagement in human-agent interactions, emphasizing its importance and indicating the growing interest of researchers in the field [1]. Backchannels like non-verbal gestures (nods and smiles), non-verbal vocalizations (mm, uh-huh, laughs) and verbal expressions (yes, right) are an important aspect of engagement and have been shown to promote engagement and interest levels of the user [2,3]. Researchers have mainly focused on rule-based back-channel generation [4,5] or data-driven unsupervised methods [6].…”

Section: Introductionmentioning

confidence: 99%

Speech Driven Backchannel Generation Using Deep Q-Network for Enhancing Engagement in Human-Robot Interaction

et al. 2019

Self Cite

View full text Add to dashboard Cite

We present a novel method for training a social robot to generate backchannels during human-robot interaction. We address the problem within an off-policy reinforcement learning framework, and show how a robot may learn to produce non-verbal backchannels like laughs, when trained to maximize the engagement and attention of the user. A major contribution of this work is the formulation of the problem as a Markov decision process (MDP) with states defined by the speech activity of the user and rewards generated by quantified engagement levels. The problem that we address falls into the class of applications where unlimited interaction with the environment is not possible (our environment being a human) because it may be time-consuming, costly, impracticable or even dangerous in case a bad policy is executed. Therefore, we introduce deep Q-network (DQN) in a batch reinforcement learning framework, where an optimal policy is learned from a batch data collected using a more controlled policy. We suggest the use of human-to-human dyadic interaction datasets as a batch of trajectories to train an agent for engaging interactions. Our experiments demonstrate the potential of our method to train a robot for engaging behaviors in an offline manner.

show abstract

Analysis of Engagement and User Experience with a Laughter Responsive Social Robot

Cited by 20 publications

References 19 publications

Engagement recognition by a latent character model based on multimodal listener behaviors in spoken dialogue

Engagement recognition by a latent character model based on multimodal listener behaviors in spoken dialogue

Multifaceted Engagement in Social Interaction with a Machine: The JOKER Project

Speech Driven Backchannel Generation Using Deep Q-Network for Enhancing Engagement in Human-Robot Interaction

Contact Info

Product

Resources

About