Building autonomous sensitive artificial listeners (Extended abstract)

Schröder, Marc; Bevacqua, Elisabetta; Cowie, Roddy; Eyben, Florian; Güneş, Hatice; Heylen, Dirk; Maat, Mark ter; McKeown, Gary; Pammi, Sathish; Pantić, Maja; Pélachaud, Catherine; Schuller, Björn; Sevin, Étienne de; Valstar, Michel; Wöllmer, Martin

doi:10.1109/acii.2015.7344610

Cited by 5 publications

(3 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using this model, the ECA was perceived as more natural; it also created more rapport with its interlocutor during the interaction. Schröder et al (2015) developed a sensitive artificial listener that was able to produce backchannels. They developed a model that predicted when an ECA should display a backchannel and with which intention.…”

Section: State Of the Artmentioning

confidence: 99%

Adaptation Mechanisms in Human–Agent Interaction: Effects on User’s Impressions and Engagement

Biancardi

Dermouche

Pélachaud

2021

Front. Comput. Sci.

Self Cite

View full text Add to dashboard Cite

Adaptation is a key mechanism in human–human interaction. In our work, we aim at endowing embodied conversational agents with the ability to adapt their behavior when interacting with a human interlocutor. With the goal to better understand what the main challenges concerning adaptive agents are, we investigated the effects on the user’s experience of three adaptation models for a virtual agent. The adaptation mechanisms performed by the agent take into account the user’s reaction and learn how to adapt on the fly during the interaction. The agent’s adaptation is realized at several levels (i.e., at the behavioral, conversational, and signal levels) and focuses on improving the user’s experience along different dimensions (i.e., the user’s impressions and engagement). In our first two studies, we aim to learn the agent’s multimodal behaviors and conversational strategies to dynamically optimize the user’s engagement and impressions of the agent, by taking them as input during the learning process. In our third study, our model takes both the user’s and the agent’s past behavior as input and predicts the agent’s next behavior. Our adaptation models have been evaluated through experimental studies sharing the same interacting scenario, with the agent playing the role of a virtual museum guide. These studies showed the impact of the adaptation mechanisms on the user’s experience of the interaction and their perception of the agent. Interacting with an adaptive agent vs. a nonadaptive agent tended to be more positively perceived. Finally, the effects of people’s a priori about virtual agents found in our studies highlight the importance of taking into account the user’s expectancies in human–agent interaction.

show abstract

Section: State Of the Artmentioning

confidence: 99%

Adaptation Mechanisms in Human–Agent Interaction: Effects on User’s Impressions and Engagement

Biancardi

Dermouche

Pélachaud

2021

Front. Comput. Sci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Additionally, he points out that back-channels can carry different functions depending on their realization. Listener behavior, including vocal back-channels and head nods, has been explored in dyadic human-agent interaction (Gustafson et al, 2005;Maatman et al, 2005;Gratch et al, 2006;Sidner et al, 2006;Douglas-Cowie et al, 2008;Huang et al, 2011;Schroder et al, 2015) and to our knowledge, at least in one case, also for multi-party human-agent interaction (Wang et al, 2013). Douglas-Cowie et al (2008) used an artificial listening agent in a wizard-of-oz setting.…”

Section: Audio-visual Feedbackmentioning

confidence: 99%

“…They found this to be a successful method to collect emotionally colored multi-modal interaction data. Schroder et al (2015) could show that their artificial emotionally expressive listening system had a positive effect on user engagement in comparison to a non-expressive baseline system. They highlighted as one of their main contributions the uniqueness of their system in that it creates a loop between multimodal human-human analysis, interpretation and affective generation of non-verbal behavior in a human-agent conversational setting.…”

Section: Audio-visual Feedbackmentioning

confidence: 99%

Towards an Engagement-Aware Attentive Artificial Listener for Multi-Party Interactions

et al. 2021

View full text Add to dashboard Cite

Listening to one another is essential to human-human interaction. In fact, we humans spend a substantial part of our day listening to other people, in private as well as in work settings. Attentive listening serves the function to gather information for oneself, but at the same time, it also signals to the speaker that he/she is being heard. To deduce whether our interlocutor is listening to us, we are relying on reading his/her nonverbal cues, very much like how we also use non-verbal cues to signal our attention. Such signaling becomes more complex when we move from dyadic to multi-party interactions. Understanding how humans use nonverbal cues in a multi-party listening context not only increases our understanding of human-human communication but also aids the development of successful human-robot interactions. This paper aims to bring together previous analyses of listener behavior analyses in human-human multi-party interaction and provide novel insights into gaze patterns between the listeners in particular. We are investigating whether the gaze patterns and feedback behavior, as observed in the human-human dialogue, are also beneficial for the perception of a robot in multi-party human-robot interaction. To answer this question, we are implementing an attentive listening system that generates multi-modal listening behavior based on our human-human analysis. We are comparing our system to a baseline system that does not differentiate between different listener types in its behavior generation. We are evaluating it in terms of the participant’s perception of the robot, his behavior as well as the perception of third-party observers.

show abstract

Emotion Components and Understanding in Humans and Machines

Bellon

2023

Technikzukünfte, Wissenschaft Und Gesellschaft / Futures of Technology, Science and Society

View full text Add to dashboard Cite

Building autonomous sensitive artificial listeners (Extended abstract)

Cited by 5 publications

References 27 publications

Adaptation Mechanisms in Human–Agent Interaction: Effects on User’s Impressions and Engagement

Adaptation Mechanisms in Human–Agent Interaction: Effects on User’s Impressions and Engagement

Towards an Engagement-Aware Attentive Artificial Listener for Multi-Party Interactions

Emotion Components and Understanding in Humans and Machines

Contact Info

Product

Resources

About