Chatbots language design: the influence of language variation on user experience

Abstract

Chatbots are often designed to mimic social roles attributed to humans. However, little is known about the impact on user's perceptions of using language that fails to conform to the associated social role. Our research draws on sociolinguistic theory to investigate how a chatbot's language choices can adhere to the expected social role the agent performs within a given context. In doing so, we seek to understand whether chatbots design should account for linguistic register. This research analyzes how register differences play a role in shaping the user's perception of the human-chatbot interaction. Ultimately, we want to determine whether register-specific language influences users' perceptions and experiences with chatbots. We produced parallel corpora of conversations in the tourism domain with similar content and varying register characteristics and evaluated users' preferences of chatbot's linguistic choices in terms of appropriateness, credibility, and user experience. Our results show that register characteristics are strong predictors of user's preferences, which points to the needs of designing chatbots with register-appropriate language to improve acceptance and users' perceptions of chatbot interactions.CCS Concepts: • Human-centered computing → Human computer interaction (HCI); Empirical studies in HCI; Natural language interfaces.

Why should we care about register? Reflections on chatbot language design

“…In this position paper, we argue that register analysis should be used as a technique for providing a theoretical basis for chatbot language design, grounded on the results presented in Chaves et al [6]. To design register-specific language for chatbots, we need to enrich chatbots with computational models that can adapt the utterances to conform with the expected register, in an effort to mimic the subconscious humans' language production process.…”

“…Previous studies have found that appropriate language style is not relevant for determining user satisfaction as long as the user can understand the chatbot's answer, only advising that the chatbot's language style should be "mildly appropriate to the service the chatbot provides" [1]. In a previous study [6], however, we found evidence that user experience goes beyond merely comprehending a chatbot's utterance to whether the user perceives the chatbot's language as appropriate and credible. In that study, we explored the applicability of register to human-chatbot interactions by developing a rationale for accounting for register in chatbot design, and providing a mechanism for implementing theory into design practice.…”

“…The work presented by Chaves et al [6] is a first step toward the applicability of register theory in chatbot language design. In the next section, we point out some research opportunities.…”

“…Then, one can empirically evaluate the extent to which the generalization to similar domains would apply. Influence of sentence structure: the study presented in Chaves et al [6] analyzed the influence of linguistic features individually without considering the position where they occur in the sentence or the frequent co-occurrences with other features. We are not aware of a methodology that automatically weighs the features according to their place in the sentence.…”