Psychological responses to simulated displays of mismatched emotional expressions

Abstract

Embodied agents are often designed with the ability to simulate human emotion. This paper investigates the psychological impact of simulated emotional expressions on computer users with a particular emphasis on how mismatched facial and audio expressions are perceived (e.g. a happy face with a concerned voice). In a within-subjects repeated measures experiment (N = 68), mismatched animations were perceived as more engaging, warm, concerned and happy when a happy or warm face was in the animation (as opposed to a neutral or concerned face) and when a happy or warm voice was in the animation (as opposed to a neutral or concerned voice). The results appear to follow cognitive dissonance theory as subjects attempted to make mismatched expressions consistent on both the visual and audio dimensions of animations, resulting in confused perceptions of the emotional expressions. Design implications for affective embodied agents are discussed and future research areas identified.

The Effects of a Pedagogical Agent’s Smiling Expression on the Learner’s Emotions and Motivation in a Virtual Learning Environment

“…This observation mirrors earlier findings by Creed and Beale (2008), which indicated that mismatched facial and voice expressions elicit negative responses from people. Furthermore, there were a few comments describing the lack of synchronization between speech and lip movement, such as "fake" and "creepy," suggesting the uncanny valley effect, which might have contributed to the negative perceptions of the agent's smile (see Tinwell, Grimshaw, Nabi, & Williams, 2011).…”

Robots that express emotion elicit better human teaching

“…A study that compared participants' impressions of a virtual agent that either was or was not consistent in its emotional expression replicated the findings about human-human impressions [9]. Participants felt a similar sense of cognitive dissonance when the virtual agent displayed inconsistent emotional expressions.…”

'n Eksploratiewe ondersoek na adolessente se internetgebruik en die identifisering van moontlike risikofaktore ten opsigte van hulle psigososiale welstand

“…Die meeste navorsers op die gebied van emosies het die grootste deel van die vorige eeu 'n kultuurteorie aangehang: Daar is geglo emosies is kultuur-spesifieke aangeleerde vorms van gedrag wat iemand net kan ervaar nadat hy of sy dit by ander mense waargeneem het (Creed & Beale 2008 • Geskrewe taal. Mense gebruik dikwels stories, gedigte en persoonlike briewe om uitdrukking aan hulle emosies te gee.…”

“…'n Mens kan ook die kleur, grootte en vorm van doi.10.17159/2224-7912/2015/v55n3a11 woorde verander en teks animeer om emosionele betekenisse oor te dra (Wang et al 2004). Simbole soos :-) vir gelukkig wees en :-( vir hartseer wees, kan ook emosies oordra, en is veral gewild in domeine waar emosionele inligting kortkom, soos e-posse, kitsboodskappe en SMS'e (Creed & Beale 2008).…”