Effects of public vs. private automated transcripts on multiparty communication between native and non-native english speakers

Abstract

Real-time transcripts generated by automated speech recognition (ASR) technologies have the potential to facilitate communication between native speakers (NS) and non-native speakers (NNS). Previous studies of ASR have focused on how transcripts aid NNS speech comprehension. In this study, we examine whether transcripts benefit multiparty real-time conversation between NS and NNS. We hypothesized that ASR transcripts would be more beneficial when the transcripts were publicly shared by all group members as opposed to when they were seen only by the NNS. To test our hypothesis, we conducted a lab experiment in which 14 groups of native and non-native speakers engaged in a story-telling task. Half of the groups received private transcripts that were available only to the NNS; the other half received publicly shared transcripts that were available to all group members. NS spoke more clearly, and both NS and NNS rated the quality of communication higher, when transcripts were publicly shared. These findings inform the design of future tools to support multilingual group communication.

Do automated transcripts help non-native speakers catch up on missed conversation in audio conferences?

“…Although our results did not reach statistical significance, they indicated a trend that ASR technology might provide additional information about the conversational content for NNS and improve their understanding. Thus, our results complement previous works on the positive effects of text transcripts on NNS comprehension [18,19,20,24] during multiparty audio conferences in their second language [8].…”

“…If NS are able and willing to coordinate and adjust their speech to the NNS ability by speaking more slowly or articulating and enunciating more clearly, some of the problems faced by NNS during audio conferences might be alleviated [1,2,3,4,8]. However, especially in groups where the majority of people are native speakers, this may fall short of expectations [1], also because conversation can move forward only between NS participants (without NNS participation).…”

“…Previous studies have demonstrated how ASR technologies can improve NNS comprehension during formal presentations and when following television programs in their second language. In a more interactive setting, where ASR transcripts are used in a live multilingual meeting, providing feedback for NS on how their speech is transcribed during real-time audio conferences may facilitate NS adaptation to the technology and improve the accuracy of the ASR transcripts [8].…”

“…An increasing number of studies have focused on alleviating the difficulties that NNS face during real-time audio conferences with ASR technology (e.g., [8]). According to previous research, text transcripts may support NNS comprehension in pre-recorded meetings when the transcripts are provided with reasonable accuracy and delay [18,19,20,24].…”

The Combined Use of Audio and Visual Media in Computer Conferencing towards Equality in Asymmetric Second-Language Conversation

“…According to the prior studies, human volunteers created more accurate transcripts than ASR during the real-time caption processing (Lasecki, Kushalnagar, &Bigham,2014). To improve the accuracy of ASR transcripts, ASR with a human volunteer editing transcript was tested and demonstrated positive effects on conversation among group of people engaging in a collaborative task (Gao, Yamashita, Hautasaari, Echenique, & Fussell, 2014). Another experiment tested whether NS highlighting on ASR transcript help NNS, and found it beneficial for NNS to communicate with NS in real-time conversation (Pan, Yamashita&Wang,2017).…”