The Behavioral and Neural Effects of Language on Motion Perception

Abstract

Abstract■ Perception does not function as an isolated module but is tightly linked with other cognitive functions. Several studies have demonstrated an influence of language on motion perception, but it remains debated at which level of processing this modulation takes place. Some studies argue for an interaction in perceptual areas, but it is also possible that the interaction is mediated by "language areas" that integrate linguistic and visual information. Here, we investigated whether language-perception interactions were specific to the language-dominant left hemisphere by comparing the effects of language on visual material presented in the right (RVF) and left visual fields (LVF). Furthermore, we determined the neural locus of the interaction using fMRI. Participants performed a visual motion detection task. On each trial, the visual motion stimulus was presented in either the LVF or in the RVF, preceded by a centrally presented word (e.g., "rise"). The word could be congruent, incongruent, or neutral with regard to the direction of the visual motion stimulus that was presented subsequently. Participants were faster and more accurate when the direction implied by the motion word was congruent with the direction of the visual motion stimulus. Interestingly, the speed benefit was present only for motion stimuli that were presented in the RVF. We observed a neural counterpart of the behavioral facilitation effects in the left middle temporal gyrus, an area involved in semantic processing of verbal material. Together, our results suggest that semantic information about motion retrieved in language regions may automatically modulate perceptual decisions about motion. ■

How prior knowledge prepares perception: Alpha-band oscillations carry perceptual expectations and influence early visual responses

“…6). Combined, our results contradict claims that knowledge affects perception only at a very late stage 49,20,50 and provide general support for predictive processing accounts of perception, positing that knowledge may feedback to modulate lower levels of perceptual processing 3,25,51 .…”

Prior probability and feature predictability interactively bias perceptual decisions

“…For instance, there is evidence that prior expectations decrease the distance to threshold for the more likely choice (Edwards, 1965), for instance, by priming relevant feature-selective sensory neurons (Kok et al, 2014; Shulman et al, 1999). However, conflicting reports have suggested that top-down signals serve to increase the rate of evidence accumulation for a particular choice (Diederich & Busemeyer, 2006), for instance, by improving the quality of signal read-out from competing pools of sensory neurons (Cravo et al, 2013).…”

“…Indeed, there is a growing body of evidence to suggest that the brain proactively facilitates perception by constructing feature-level models or templates of expected stimuli (Clark, 2013; Summerfield & Egner, 2009). Neural correlates of predictive stimulus templates have been observed in visual (Jiang, Summerfield, & Egner, 2013; Kok, Failing, & de Lange, 2014; Summerfield, Egner, Greene, et al, 2006; White, Mumford, & Poldrack, 2012), auditory (Chennu et al, 2013), somatosensory (Carlsson, Petrovic, Skare, Petersson, & Ingvar, 2000), and olfactory (Zelano, Mohanty, & Gottfried, 2011) cortex, highlighting feature prediction as a fundamental property of perception. It follows that feature-level predictions are compared with incoming sensory signals to determine a match between the expected and observed inputs.…”

“…These studies have found compelling evidence that expectations bias sensory representations both prior to and during evidence accumulation via recurrent communication between early feature-selection and higher-order cognitive processing domains. For instance, human neuroimaging studies have detected anticipatory signals or “baseline shifts” in sensory regions that are selective for expected stimuli (Esterman & Yantis, 2010; Kok et al, 2014; Shulman et al, 1999) as well as enhanced contrast between early target and distractor representations (Jiang, Summerfield, & Egner, 2013; Kok et al, 2012). Hierarchical models of perception, such as predictive coding, propose that these early sensory modulations represent a feature-level template of the expected stimulus, generated by top-down inputs from higher-order regions that encode more abstract predictions (i.e., category probability).…”

Crossmodal Modulation of Spatial Localization by Mimetic Words

“…Does such a verbal cue affect the forward displacement of a moving object like nonverbal auditory cues (e.g., tone)? The language processing influences visual motion perception such as detection of low-level motion (Meteyard, Bahrami, & Vigliocco, 2007) and motion discrimination (Francken, Kok, Hagoort, & de Lange, 2015; Francken, Meijs, Hagoort, van Gaal, & de Lange, 2015; Pavan, Skujevskis, & Baggio, 2013). In the experiments in these studies, verbs representing directional movement (e.g., sink) were presented before a visual motion stimulus appeared, and the motion verbs congruent or incongruent with visual motion direction affected the error rate and reaction time of motion discrimination (e.g., Meteyard et al., 2007).…”