I Feel You: The Design and Evaluation of a Domotic Affect-Sensitive Spoken Conversational Agent

Abstract

We describe the work on infusion of emotion into a limited-task autonomous spoken conversational agent situated in the domestic environment, using a need-inspired task-independent emotion model (NEMO). In order to demonstrate the generation of affect through the use of the model, we describe the work of integrating it with a natural-language mixed-initiative HiFi-control spoken conversational agent (SCA). NEMO and the host system communicate externally, removing the need for the Dialog Manager to be modified, as is done in most existing dialog systems, in order to be adaptive. The first part of the paper concerns the integration between NEMO and the host agent. The second part summarizes the work on automatic affect prediction, namely, frustration and contentment, from dialog features, a non-conventional source, in the attempt of moving towards a more user-centric approach. The final part reports the evaluation results obtained from a user study, in which both versions of the agent (non-adaptive and emotionally-adaptive) were compared. The results provide substantial evidences with respect to the benefits of adding emotion in a spoken conversational agent, especially in mitigating users' frustrations and, ultimately, improving their satisfaction.

Computationally modeling interpersonal trust

“…As such, rather than exploring and comparing the predictive accuracies of various machine learning algorithms, we focus on support vector machines (SVMs) as the primary tool for our feature-focused investigation. SVMs were chosen for their wide use and prior success in modeling human behavior (Rienks et al, 2006 ; Kapoor et al, 2007 ; Jayagopi et al, 2009 ).…”

“…In similar work, Kaliouby and Robinson ( 2004 ) used a dynamic Bayesian network model to infer a person's mental state of agreement, disagreement, concentration, interest, or confusion by observing only facial expressions and head movements. Other research has tried to model cognitive states, like frustration (Kapoor et al, 2007 ), and social relations like influence and dominance among groups of people (Jayagopi et al, 2009 ; Pan et al, 2011 ). However, this article describes the first work toward computationally predicting the trusting behavior of an individual toward a social partner.…”

Cross-Cultural Perception of Spanish Synthetic Expressive Voices Among Asians

“…Prosody (intensity, vocal pitch, rhythm, rate of utterance) in speech plays a major role in expressing emotions and can be intentionally modified to communicate different feelings [43]. In this paper, we investigated the expressive synthetic voices cross which was perceived by different ethnic groups using two different features (duration and intensity).…”

Towards Truly Affective AAL Systems

“…One of the goals for computational emotion modeling is to enrich the architecture of intelligent systems with emotion mechanisms similar to those of humans, and thus endow them with the capacity to "have" emotions. In the context of AAL, some studies exist in this direction, e.g., in [16] authors describe need-inspired emotion model applied in a HiFi agent whose emotions are generated by evaluating the situation and comparing it to agent's different needs.…”