Evaluation of Four Designed Virtual Agent Personalities

Abstract

Abstract-Convincing conversational agents require a coherent set of behavioural responses that can be interpreted by a human observer as indicative of a personality. This paper discusses the continued development and subsequent evaluation of virtual agents based on sound psychological principles. We use Eysenck's theoretical basis to explain aspects of the characterization of our agents, and we describe an architecture where personality affects the agent's global behaviour quality as well as their backchannel productions. Drawing on psychological research, we evaluate perception of our agents' personalities and credibility by human viewers (N=187). Our results suggest that we succeeded in validating theoretically grounded indicators of personality in our virtual agents, and that it is feasible to place our characters on Eysenck's scales. A key finding is that the presence of behavioural characteristics reinforces the prescribed personality profiles that are already emerging from the still images. Our long-term goal is to enhance agents' ability to sustain realistic interaction with human users, and we discuss how this preliminary work may be further developed to include more systematic variation of Eysenck's personality scales.

Two Techniques for Assessing Virtual Agent Personality

“…Many studies use a within-subjects design where participants are shown multiple agents with different target personality traits, including those that are meant to be the opposites of each other (e.g. extraverted and introverted) [6,8,19,22,44,48]. While this method maximizes statistical power and is more logistically efficient, a within-subjects design may introduce unintended contrast effects.…”

“…emotional stability through gesture in virtual agents has examined the role of gesture and posture in personality attribution by human observers, though they generally tested multiple expressive modalities simultaneously (such as facial expression and gesture [6] or utterance and gesture [18]) rather than examining gesture and posture in isolation.…”

“…Increasingly sophisticated virtual agents are now sometimes designed to mimic these cues in order to induce human observers to make similar attributions about the agent with whom they are interacting. The generation of consistent, human-like patterns of expression based on prescribed personality traits makes interaction with these agents easier for users [5,6] and similarly allows for the beneficial tailoring of agents to specific users or to specific task domains [7][8][9][10][11][12][13][14]. Furthermore, there is evidence that suggests that these cues are most effective when cues are congruent, rather than conflicting [12,15] .…”

“…Personality in virtual agents can be expressed through a number of different cues, such as language usage [13,[16][17][18], facial expression [6,8,[19][20][21], and non-verbal behaviors such as gesture and posture [6,18,22,23]. For example, extraversion is associated with a faster rate of speech and larger gestures, so agents portraying extraversion talk faster and gesture with wider arms than agents portraying introversion.…”

Turing's menagerie: Talking lions, virtual bats, electric sheep and analogical peacocks: Common ground and common interest are necessary components of engagement

“…Towards the end of the project there was an evaluation phase that involved testing the quality of interactions with the Sensitive Artificial Listeners that had been created as part of the project. The SEMAINE system built four embodied conversational agents based around four characters with different personalities [6]. Participants interacted with each of these personalities in turn and had to evaluate the nature of the interaction.…”