Managing an Agent's Self-Presentational Strategies During an Interaction

Abstract

In this paper we present a computational model for managing the impressions of warmth and competence (the two fundamental dimensions of social cognition) of an Embodied Conversational Agent (ECA) while interacting with a human. The ECA can choose among four different self-presentational strategies eliciting different impressions of warmth and/or competence in the user, through its verbal and non-verbal behavior. The choice of the non-verbal behaviors displayed by the ECA relies on our previous studies. In our first study, we annotated videos of human-human natural interactions of an expert on a given topic talking to a novice, in order to find associations between the warmth and competence elicited by the expert's non-verbal behaviors (such as type of gestures, arms rest poses, smiling). In a second study, we investigated whether the most relevant non-verbal cues found in the previous study were perceived in the same way when displayed by an ECA. The computational learning model presented in this paper aims to learn in real-time the best strategy (i.e., the degree of warmth and/or competence to display) for the ECA, that is, the one which maximizes user's engagement during the interaction. We also present an evaluation study, aiming to investigate our model in a real context. In the experimental scenario, the ECA plays the role of a museum guide introducing an exposition about video games. We collected data from 75 visitors of a science museum. The ECA was displayed in human dimension on a big screen in front of the participant, with a Kinect on the top. During the interaction, the ECA could adopt one of 4 self-presentational strategies during the whole interaction, or it could select one strategy randomly for each speaking turn, or it could use a reinforcement learning algorithm to choose the strategy having the highest reward (i.e., user's engagement) after each speaking turn.

Engagement in Human-Agent Interaction: An Overview

“…Social engagement very often also includes an affective component (see for example Corrigan et al, 2016 ; Biancardi et al, 2019a ; Sohail et al, 2019 ; Youssef et al, 2019 ). The definition of the affective component of engagement often remains vague.…”

“…Similarly also Youssef et al ( 2019 ) rely on several levels of engagement annotation including both the affective as well as the (non-)verbal cue level that includes head rotation and eye-gaze. Finally, Biancardi et al ( 2019a ) combines affective- with attention- and cognitive engagement components in their detection model. Every person might of course differ to a certain degree in the way that he/she expresses the different forms of engagement.…”

“…It appears that situations and scenarios do vary considerably across studies. Examples, of such scenarios include museum guides (Pitsch et al, 2009 ; Salam and Chetouani, 2015 ; Biancardi et al, 2019a ), games (Díaz et al, 2011 ; Klotz et al, 2011 ; Leite et al, 2014 ), hospitality (Sidner and Lee, 2003 ; Sidner and Dzikovska, 2005 ), education (Leyzberg et al, 2014 ; Papadopoulos et al, 2016 ), sales (Ishii et al, 2011 ), and receptionist/direction giving (Michalowski et al, 2006 ; Bohus and Horvitz, 2014 ). With regards to museum guides, the overarching goal in the studies has been on making visitors more interested in the artwork and to provide information on the artwork on demand.…”

“…Each of these mechanisms have been implemented in the same agent architecture but affect different modules. Adaptation of the conversational strategies is done within the dialog module described in Biancardi et al ( 2019a ). The adaptation at the non-verbal level is done in the behavior generation module of the Greta platform, as described by Biancardi et al ( 2019b ).…”

“…During an interaction with a user, the agent will optimize each of the adaptation mechanisms by relying on either reinforcement learning or LSTM methods. Biancardi et al ( 2019a ) conducted an experiment in a science museum where the agent played the role of a museum guide to validate their adaptation models. The agent that adapted its behavior to maximize user's engagement was perceived as warm by participants, but they did not find any effect of agent's adaptation on users' evaluation of their experience of the interaction.…”