When a Talking-Face Computer Agent is Half-Human and Half-Humanoid: Human Identity and Consistency Preference

Abstract

Computer-generated anthropomorphic characters are a growing type of communicator that is deployed in digital communication environments. An essential theoretical question is how people identify humanlike but clearly artificial, hence humanoid, entities in comparison to natural human ones. This identity categorization inquiry was approached under the framework of consistency and tested through examining inconsistency effects from mismatching categories. Study 1 (N = 80), incorporating a self-disclosure task, tested participants' responses to a talking-face agent, which varied in four combinations of human versus humanoid faces and voices. In line with the literature on inconsistency, the pairing of a human face with a humanoid voice or a humanoid face with a human voice led to longer processing time in making judgment of the agent and less trust than the pairing of a face and a voice from either the human or the humanoid category. Female users particularly showed negative attitudes toward inconsistently paired talking faces. Study 2 (N = 80), using a task that stressed comprehension demand, replicated the inconsistency effects on judging time and females' negative attitudes but not for comprehension-related outcomes. Voice clarity overshadowed the consistency concern for comprehension-related responses. The overall inconsistency effects suggest that people treat humanoid entities in a different category from natural human ones.

The Influence of Synthetic Voice on the Evaluation of a Virtual Character

“…We find that the voice conditions have a statistically significant effect on voice trait evaluations, in agreement with previous studies, e.g., [6]. However, we find that they have little impact on the character level judgments beyond those that are voice focused.…”

“…Whilst we found natural recordings were judged as superior to synthetic speech, echoing previous work [6], we did not find a consistency effect for character evaluations seen in [6]. This may be explained by our findings around voice consistency.…”

“…In a study observing people's perceptions of characters that recited book reviews, participants rated their understanding more highly when it was given in a human rather than humanoid voice, regardless of the character's graphical representation [6]. Voice, rather than both visuals and voice, seems to be the main dimension that people consider when making judgments about understanding of the content being delivered by the character.…”

“…Research on the interplay between visuals and speech in character evaluation [6] shows that a mismatch between humanness of the voice and a character's graphical representation can negatively affect people's perceptions towards a character. Whenever people interacted with an avatar that had an audio-visual mismatch (e.g., either a virtual humanoid avatar with a human voice or a video of a human with a humanoid voice), they trusted the character less and had lower ratings of self-disclosure than when the character's visual representation matched the voice used.…”

“…Although there seems to be an interaction between visuals and speech in character level judgments, speech tends to be a more significant driver of comprehension based judgments [6]. In a study observing people's perceptions of characters that recited book reviews, participants rated their understanding more highly when it was given in a human rather than humanoid voice, regardless of the character's graphical representation [6].…”