Multimodal Backchannels for Embodied Conversational Agents

Bevacqua, Elisabetta; Pammi, Sathish; Hyniewska, Sylwia; Schröder, Marc; Pélachaud, Catherine

doi:10.1007/978-3-642-15892-6_21

Cited by 41 publications

(29 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by these developments, a number of recent systems use vocal cues to trigger the behaviour of virtual agents. For example, in (Bevacqua et al, 2010) vocal cues are used to generate backchannels (i.e., non-intrusive signals provided during the speaker's turn). Acosta and Ward proposed a system that uses speech and prosody variation to build rapport between human and agent (Acosta and Ward, 2011), and Cavazza et al used vocal signals for character-based interactive storytelling (Cavazza et al, 2009).…”

Section: Emotions In Vocal Signalsmentioning

confidence: 99%

Towards Interactive Agents that Infer Emotions from Voice and Context Information

Formolo¹,

Bosse²

2017

EAI Endorsed Transactions on Creative Technologies

View full text Add to dashboard Cite

Conversational agents are increasingly being used for training of social skills. One of their most important benefits is their ability to understand the user`s emotions, to be able to provide natural interaction with humans. However, to infer a conversation partner's emotional state, humans typically make use of contextual information as well. This work proposes an architecture to extract emotions from human voice in combination with the context imprint of a particular situation. With that information, a computer system can achieve a more human-like type of interaction. The architecture presents satisfactory results. The strategy of combining 2 algorithms, one to cover 'common cases' and another to cover 'borderline cases' significantly reduces the percentage of mistakes in classification. The addition of context information also increases the accuracy in emotion inferences.

show abstract

Section: Emotions In Vocal Signalsmentioning

confidence: 99%

Towards Interactive Agents that Infer Emotions from Voice and Context Information

Formolo¹,

Bosse²

2017

EAI Endorsed Transactions on Creative Technologies

View full text Add to dashboard Cite

show abstract

“…We therefore expect that a different type of backchannel, produced in the same structural context, will be perceived differently by human observers. Although some researchers have addressed the perception of different types of backchannels [22,23] in isolation, none of them have investigated their perception in a conversational context. Therefore, in this paper, we investigate how the quantity, type and timing of backchannels influences how human-like the backchannel behavior is perceived by human observers.…”

Section: Related Workmentioning

confidence: 99%

Backchannels: Quantity, Type and Timing Matters

Poppe

Truong

Heylen

2011

Intelligent Virtual Agents

View full text Add to dashboard Cite

Abstract. In a perception experiment, we systematically varied the quantity, type and timing of backchannels. Participants viewed stimuli of a real speaker side-by-side with an animated listener and rated how human-like they perceived the latter's backchannel behavior. In addition, we obtained measures of appropriateness and optionality for each backchannel from key strokes. This approach allowed us to analyze the influence of each of the factors on entire fragments and on individual backchannels. The originally performed type and timing of a backchannel appeared to be more human-like, compared to a switched type or random timing. In addition, we found that nods are more often appropriate than vocalizations. For quantity, too few or too many backchannels per minute appeared to reduce the quality of the behavior. These findings are important for the design of algorithms for the automatic generation of backchannel behavior for artificial listeners.

show abstract

“…To solve this problem, a number of models have been proposed for determining the appropriate timing of feedback (ranging from rules-based to complex machine learning approaches, e.g., [25,17]) and for turning different feedback functions into nonverbal as well as vocal and linguistic behaviour [24,22,5]. Less attention has been paid to the question which feedback function to use (exceptions being [12,14,4]), mainly due to the open challenge of understanding unrestricted spoken language in large domains, which would lead agents to give frequent and less informative signals of non-understanding.…”

Section: Communicative Feedback In Human-agent Interactionmentioning

confidence: 99%

“…Researchers in the virtual agents community have noticed the importance of these mechanisms and have started to develop systems that act as 'active listeners', i.e., agents that produce feedback signals in response to user actions [12,14,17,4,5]. In contrast to this, the at least equally important capability of being able to perceive, interpret, and respond to communicative user feedback is effectively non-existent in conversational virtual agents (but see [19] for a first effort).…”

Section: Introductionmentioning

confidence: 99%

Towards Conversational Agents That Attend to and Adapt to Communicative User Feedback

Buschmeier

Kopp

2011

Intelligent Virtual Agents

View full text Add to dashboard Cite

Successful dialogue is based on collaborative efforts of the interactants to ensure mutual understanding. This paper presents work towards making conversational agents 'attentive speakers' that continuously attend to the communicative feedback given by their interlocutors and adapt their ongoing and subsequent communicative behaviour to their needs. A comprehensive conceptual and architectural model for this is proposed and first steps of its realisation are described. Results from a prototype implementation are presented.

show abstract

Multimodal Backchannels for Embodied Conversational Agents

Cited by 41 publications

References 9 publications

Towards Interactive Agents that Infer Emotions from Voice and Context Information

Towards Interactive Agents that Infer Emotions from Voice and Context Information

Backchannels: Quantity, Type and Timing Matters

Towards Conversational Agents That Attend to and Adapt to Communicative User Feedback

Contact Info

Product

Resources

About