Improving Multilingual Collaboration by Displaying How Non-native Speakers Use Automated Transcripts and Bilingual Dictionaries

Abstract

Conversational grounding, or establishing mutual knowledge that messages have been understood as intended, can be difficult to achieve when some conversational participants are using a non-native language. These difficulties in grounding can be challenging for native speakers to detect. In this paper, we examine the value of signaling potential grounding problems to native speakers (NS) by displaying how non-native speakers (NNS) use automated transcripts and bilingual dictionaries. We conducted a laboratory experiment in which NS and NNS of English collaborated via audio conferencing on a map navigation task. Triads of one NS guider, one NS follower, and one NNS follower performed the task using one of three awareness displays: (a) a no awareness display that showed only the automated transcripts, (b) a general awareness display that showed whether each follower was reading the automated transcripts and/or translating a word; or (c) a detailed awareness display that showed which line of the transcripts a follower was reading and/or which words he/she was translating. NS guiders and NNS followers collaborated most successfully with the detailed awareness display, while NS guiders and NS followers performed equally across conditions. Our findings suggest several ways to improve systems to support multilingual collaboration.

Effects of Automated Transcripts on Non-Native Speakers' Listening Comprehension

“…We used them as a base because they also focus on the listening comprehension problems of NNSs that occur during their cognitive processing of speech input. Note that we added a new category "lack of vocabulary" to the previous categories [8], [11] because it can be solved by adding a dictionary function to the ASR transcripts [12]. All the interview data were coded independently by two coders, and all discrepancies were discussed until an agreement was reached.…”

Increasing Native Speakers' Awareness of the Need to Slow Down in Multilingual Conversations Using a Real-Time Speech Speedometer

“…Duan and colleagues [13] employed an intelligent agent that asked clarification questions so that NS were aware of potential confusions that NNS may be too shy to acknowledge. Gao et al [19] provided NS with a shared display of how their NNS teammates were using real-time transcripts and a bilingual dictionary and found that this increased NS's awareness of the specific difficulties NNS were experiencing. Still, these tools engaged NS only at the awareness level.…”

“…No significant effect of trial (F [1, 38] = 1.75, p = . 19) and no interaction (F [1, 38] = 1.75, p = .19) was found. Contrary to H3b, participants perceived lower clarity of NS speech when NS used the Speech Speedometer (M=5.28, SE=.33) than when they used the control display (M=6.00, SE=.27).…”

Identifying Support Opportunities for Foreign Students: Disentangling Language and Non-language Problems Among a Unique Population

“…Previous work has explored methods for improving intercultural communication between native and non-native speakers. This includes efforts to reduce cognitive burden [39] and improve comprehension [40,41] for non-native speakers, and to establish conversational grounding in multilingual conversations [42][43][44]. For the most part, evaluations of these kinds of tools are based on how participants perform communication tasks in lab settings.…”

“…Foreign students would benefit from tools that help them communicate their difficulties to host-nationals. With regard to language-difficulties, previous work has demonstrated that making native-speakers aware of non-native speakers' difficulties encouraged more effective communication strategies [42]. Additionally, language barriers tend to be asymmetric because, although native speakers are aware of non-native speakers' difficulties, they do not understand how this affect nonnative speakers' behavior [60].…”