The role of visual representations during the lexical access of spoken words

Abstract

Do visual representations contribute to spoken word recognition? We examine, using MEG, the effects of sublexical and lexical variables at superior temporal (ST) areas and the posterior middle temporal gyrus (pMTG) compared with that of word imageability at visual cortices. Embodied accounts predict early modulation of visual areas by imageability - concurrently with or prior to modulation of pMTG by lexical variables. Participants responded to speech stimuli varying continuously in imageability during lexical decision with simultaneous MEG recording. We employed the linguistic variables in a new type of correlational time course analysis to assess trial-by-trial activation in occipital, ST, and pMTG regions of interest (ROIs). The linguistic variables modulated the ROIs during different time windows. Critically, visual regions reflected an imageability effect prior to effects of lexicality on pMTG. This surprising effect supports a view on which sensory aspects of a lexical item are not a consequence of lexical activation.

A task-dependent causal role for low-level visual processes in spoken word comprehension.

“…For instance, in a continuous flash suppression study we recently found evidence that passive listening of object words activates visual processes involved in a task as basic as visual detection (Ostarek & Huettig, in press). Similarly, a recent magnetoencephalography study (Lewis & Poeppel, 2014) using an auditory LDT found a correlation between imageability and activity in visual association cortex within the first 200 ms after word onset. In the light of findings like these, we do not intend to rule out an involvement of the visual system in the LDT and word class decision experiments, but we can tell from the data that it did not have a functional role in these tasks.…”

Visual and Motor Cortices Differentially Support the Translation of Foreign Language Words

“…In particular, it integrates well with research into the representation of words in the native language. Lewis and Poeppel [26], for example, demonstrated that native language words with a high amount of visual associations (such as the word ''apple'') can elicit responses in visual areas even before the lexical access to the word is complete, suggesting that sensory areas are involved in the early analyses of words and not only in late imagery processes. Hauk et al [24] showed that hearing action words activated the motor cortex somatotopically, and Pulvermuller et al [27] demonstrated that these activations are behaviorally relevant: inhibitory transcranial magnetic stimulation applied to arm-and leg-motion-controlling parts of the motor cortex led to slower recognition of words referring to arm and leg motion, respectively.…”

Cortical Activity Predicts Which Older Adults Recognize Speech in Noise and When

“…This association was primarily driven by reduced linearity of diffusion tensors, which can be affected by the density of axon membranes (Alexander et al, 2000;Beaulieu, 2002;Mori and Zhang, 2006). Therefore, age-related declines in the ability to maintain attention (Lustig et al, 2001;Tun et al, 2009) during speech recognition in noise are predicted to stem, at least in part, from IFOF declines.…”

“…Older adults also have exhibited increased visual cortex activity while performing speech recognition and other listening tasks (Fernandes et al, 2006;Eckert et al, 2008;Peiffer et al, 2009;. This activity may support word recognition because early visual cortex activity encodes category information for sounds (Vetter et al, 2014) and is sensitive to the imageability of spoken words (Lewis and Poeppel, 2014).…”