Wow, You Are Terrible at This!

Abstract

While the effects of virtual agents in terms of likeability, uncanniness, etc. are well explored, it is unclear how their appearance and the feedback they give affects people's reactions. Is critical feedback from an agent embodied as a mouse or a robot taken less serious than from a human agent? In an intercultural study with 120 participants from Germany and the US, participants had to find hidden objects in a game and received feedback on their performance by virtual agents with different appearances. As some levels were designed to be unsolvable, critical feedback was unavoidable. We hypothesized that feedback would be taken more serious, the more human the agent looked. Also, we expected the subjects from the US to react more sensitively to criticism. Surprisingly, our results showed that the agents' appearance did not significantly change the participants' perception. Also, while we found highly significant differences in inspirational and motivational effects as well as in perceived task load between the two cultures, the reactions to criticism were contrary to expectations based on established cultural models. This work improves our understanding on how affective virtual agents are to be designed, both with respect to culture and to dialogue strategies. CCS CONCEPTS • Computing methodologies → Intelligent agents; • Social and professional topics → Cultural characteristics; • Humancentered computing → User studies.

Conversational agents: a survey on culturally informed design practices

“…Similarly, Löffler et al (2021) used a discursive design approach to create the protestant blessing robot BlessU2 by arranging emerging technological elements to hypothesize future products and gain a better understanding of their potential implications in a given context. Lastly, Wang et al (2020) developed a game where players ask for support from a virtual agent.…”

“…Evaluation of the conversational agent can discuss qualities of the bot in terms of satisfaction (Yadav et al, 2019), usefulness (Wang et al, 2020; Yadav et al, 2019, closeness to the user (Wang et al, 2020), trustworthiness (Wang et al, 2020), appropriateness (Wang et al, 2020), likeability (Wang et al, 2020; Rahman et al, 2021; Löffler et al, 2021; Pataranutaporn et al, 2019; Trovato et al, 2017; Carnell and Lok, 2018; Liao and He, 2020, perceived intelligence (Trovato et al, 2017) or if the participant would continue to use after the study (Wang et al, 2020; Guerrero-Vásquez et al, 2020; Yadav et al, 2019; Liao and He, 2020. Other more specific evaluation of the interaction includes understanding the adequacy of appearance, identity, language, behavior, personality, friendliness, and preference between conversational agents.…”

“…Surveys can include open-ended questions, the Likert scale, the Semantic scale (e.g., Creppy to Pleasant, Scary to Innocuous, among others) (Trovato et al, 2017), Task load scale (Wang et al, 2020), or the attribution of emotion as a way to measure their perception with a robot (Wang et al, 2020; Löffler et al, 2021. Other examples include the use of the Method for the Assessment of Experience (MAX), a post-use evolution to measure general experiences (emotion, ease of use, usefulness, and intention to use), or the Perceived Interpersonal Closeness Scale (PICS).…”

“…Rehm et al (2018) emphasize that cultural theories or frameworks make predictions about behavior and are often stereotypical. Largely used, Hofstede's model can present issues upon its use on cultural tailoring (Wang et al, 2020). Wang et al (2020) argue that using Hofstede's model may help designers improve familiarity by mimicking or absorbing cultural markers and promoting values such as likeability, trust, and empathy.…”

“…Largely used, Hofstede's model can present issues upon its use on cultural tailoring (Wang et al, 2020). Wang et al (2020) argue that using Hofstede's model may help designers improve familiarity by mimicking or absorbing cultural markers and promoting values such as likeability, trust, and empathy. However, more is needed to task-oriented and complex scenarios.…”