Semantic information mediates visual attention during spoken word recognition in Chinese: Evidence from the printed-word version of the visual-world paradigm

Abstract

In the present study, we investigated whether the activation of semantic information during spoken word recognition can mediate visual attention's deployment to printed Chinese words. We used a visual-world paradigm with printed words, in which participants listened to a spoken target word embedded in a neutral spoken sentence while looking at a visual display of printed words. We examined whether a semantic competitor effect could be observed in the printedword version of the visual-world paradigm. In Experiment 1, the relationship between the spoken target words and the printed words was manipulated so that they were semantically related (a semantic competitor), phonologically related (a phonological competitor), or unrelated (distractors). We found that the probability of fixations on semantic competitors was significantly higher than that of fixations on the distractors. In Experiment 2, the orthographic similarity between the spoken target words and their semantic competitors was manipulated to further examine whether the semantic competitor effect was modulated by orthographic similarity. We found significant semantic competitor effects regardless of orthographic similarity. Our study not only reveals that semantic information can affect visual attention, it also provides important new insights into the methodology employed to investigate the semantic processing of spoken words during spoken word recognition using the printed-word version of the visualworld paradigm.Keywords Semantic processing . Printed-word paradigm . Spoken word recognition . Chinese . Eye movements Examining the types of information involved in spoken word recognition and their effects on visual attention can help us better understand how information from the visual and auditory modalities interacts to influence eye movements. In the present study, we investigated whether the activation of semantic information in spoken word recognition can mediate the deployment of visual attention to printed Chinese words.

The time course of morphological processing during spoken word recognition in Chinese

“…The results did not change after these four items were excluded from analyses. This finding was in line with theShen et al (2016), which found that the semantic competitor effect was not modulated by orthographic similarity in Chinese.Psychon BullRev (2017Rev ( ) 24:1957Rev ( -1963…”

“…This paradigm can provide close time-locked measures and reflect online cognitive processing with spoken words unfolding over time (Huettig & McQueen, 2007). Shen, Qu, and Li (2016) showed that a printed-word paradigm is applicable for use in investigating semantic information processing during Chinese spoken word recognition. They instructed participants to listen to a spoken target word embedded in a neutral sentence and view a printed word visual display, including a semantic competitor (i.e., the word was semantically related to the spoken target word) and two unrelated distractors.…”

“…Shen et al (2016) showed that preview time of the visual display did not influence semantic information processing, and hence only a short preview condition was included in this study.…”

Visual attention shift to printed words during spoken word recognition in Chinese: The role of phonological information

“…However, the role of phonological information was not as stable as the semantic information in such study. Shen et al (2016) found that the phonological competitor effect was only significant in the short-preview condition (i.e., visual display was previewed for 200 ms) in Experiment 2 but was insignificant with another set of experimental materials in the same condition in Experiment 1. Another study conducted by Meng (2014), using a similar experimental design, found no phonological competitor effect when the phonological competitor shared the initial phonological information with target words.…”

“…We investigate whether phonological information at full-phonological overlap could mediate the shift of visual attention to printed Chinese words. Shen, Qu, and Li (2016) found that both semantic and phonological information could guide visual attention shift to printed words. However, the role of phonological information was not as stable as the semantic information in such study.…”