Attention-Sharing Initiative of Multimodal Processing in Simultaneous Interpreting

Abstract

This study sets out to describe simultaneous interpreters' attention-sharing initiatives when exposed under input from both videotaped speech recording and real-time transcriptions. Separation of mental energy in acquiring visual input accords with the human brain's statistic optimization principle where the same property of an object is presented through diverse fashions. In examining professional interpreters' initiatives, the authors invited five professional English-Chinese conference interpreters to simultaneously interpret a videotaped speech with real-time captions generated by speech recognition engine while meanwhile monitoring their eye movements. The results indicate the professional interpreters' preferences in referring to visually presented captions along with the speaker's facial expressions, where low-frequency words, proper names, and numbers gained greater attention than words with higher frequency. This phenomenon might be explained by the working memory theory in which the central executive enables redundancy gains retrieved from dual-channel information.

Multimodal Interpretation Studies in China

“…For instance, interpreters can obtain diversified visual information, such as the speaker's facial expressions, gestures, head movements, lip movements, audience, venue layout, and PowerPoint and images, etc. Thus, interpreting is a typical multimodal activity (Seeber, 2017;Boria & Tomalin, 2020;Li & Fan, 2020), requiring interpreters to acquire useful messages through multiple senses and different symbolic resources, and then convey processed information in another language.…”