Effects of language proficiency on eye-gaze in second language conversations

Umata, Ichiro; Yamamoto, Seiichi; Ijuin, Koki; Nishida, Masafumi

doi:10.1145/2522848.2522868

Cited by 9 publications

(4 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For such problems, based on ASR transcripts where NNSs pressed a button, automatically providing dictionary definitions and images might be helpful. Previous work suggested that eye tracking is not only useful for analyzing user behavior, but it can also be used as an input mechanism and a means of interacting with a program, a game, or some other technology [18], [19]. In our study, we observed some typical eye movements of NNSs when encountering comprehension problems: 1) fixating on a word or phrase (Fig.…”

Section: Introducing Other Technologies To Supplement Asr Transcriptsmentioning

confidence: 60%

Effects of Automated Transcripts on Non-Native Speakers' Listening Comprehension

Cao

Yamashita

Ishida

2018

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

Previous research has shown that transcripts generated by automatic speech recognition (ASR) technologies can improve the listening comprehension of non-native speakers (NNSs). However, we still lack a detailed understanding of how ASR transcripts affect listening comprehension of NNSs. To explore this issue, we conducted two studies. The first study examined how the current presentation of ASR transcripts impacted NNSs' listening comprehension. 20 NNSs engaged in two listening tasks, each in different conditions: C1) audio only and C2) audio+ASR transcripts. The participants pressed a button whenever they encountered a comprehension problem, and explained each problem in the subsequent interviews. From our data analysis, we found that NNSs adopted different strategies when using the ASR transcripts; some followed the transcripts throughout the listening; some only checked them when necessary. NNSs also appeared to face difficulties following imperfect and slightly delayed transcripts while listening to speech-many reported difficulties concentrating on listening/reading or shifting between the two. The second study explored how different display methods of ASR transcripts affected NNSs' listening experiences. We focused on two display methods: 1) accuracy-oriented display which shows transcripts only after the completion of speech input analysis, and 2) speed-oriented display which shows the interim analysis results of speech input. We conducted a laboratory experiment with 22 NNSs who engaged in two listening tasks with ASR transcripts presented via the two display methods. We found that the more the NNSs paid attention to listening to the audio, the more they tended to prefer the speed-oriented transcripts, and vice versa. Mismatched transcripts were found to have negative effects on NNSs' listening comprehension. Our findings have implications for improving the presentation methods of ASR transcripts to more effectively support NNSs.

show abstract

Section: Introducing Other Technologies To Supplement Asr Transcriptsmentioning

confidence: 60%

Effects of Automated Transcripts on Non-Native Speakers' Listening Comprehension

Cao

Yamashita

Ishida

2018

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…We used a multimodal corpus [12], [13] of conversations by three participants in these analyses. The main features of the corpus are briefly explained as follows.…”

Section: Multimodal Corpusmentioning

confidence: 99%

“…Previous research [12], [13] has suggested that speakers are gazed at more by listeners in L2 conversations than in L1 conversations. We used the listener's gazing ratio to analyze the eye gaze activities of listeners while the other participants were speaking.…”

Section: Analysis 1: Listener's Eye Gaze Activities During Utterancesmentioning

confidence: 99%

“…Our previous research also revealed that the participants gazed more at speakers in conversations in L2 than those in L1 when they joined small-party conversations [11], [12], [13]. Several possible reasons have arisen to explain the differences between Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Eye Gaze Analyses in L1 and L2 Conversations

Ijuin

Taguchi

Umata

et al. 2014

Proceedings of the 2014 Workshop on Understanding and Modeling Multiparty, Multimodal Interactions

Self Cite

View full text Add to dashboard Cite

The importance of conversation in a second language (L2) during international collaboration continues to increase, but the features of non-verbal communications such as eye gaze and gestures in L2 conversation have not been clarified to the extent they have in the native language (L1). This study provides quantitative analyses of eye gaze activities to examine their differences in conversations between L1 and L2. These analyses clarify that the frequency of utterances during which listeners gaze at speakers throughout the utterance is greater in L2 than in L1 conversations when they join small-party conversations.

show abstract