The context-contingent nature of cross-modal activations of the visual cortex

Abstract

Real-world environments are nearly always multisensory in nature. Processing in such situations confers perceptual advantages, but its automaticity remains poorly understood. Automaticity has been invoked to explain the activation of visual cortices by laterally-presented sounds. This has been observed even when the sounds were task-irrelevant and spatially uninformative about subsequenttargets. An auditory-evoked contralateral occipital positivity (ACOP) at ~250ms post-sound onset has been postulated as the event-related potential (ERP) correlate of this cross-modal effect. However, the spatial dimension of the stimuli was nevertheless relevant in all prior studies where the ACOP was observed. By manipulating the implicit predictability of the location of lateralised sounds in a passive auditory paradigm, we tested the automaticity of cross-modal activations of visual cortices.128-channel ERP data from healthy participants were analysed within an electrical neuroimaging framework. The timing, topography, and localisation resembled previous characterisations of the ACOP. However, the cross-modal activations of visual cortices by sounds were critically dependent on whether the sound location was (un)predictable. Our results are the first direct evidence that this particular cross-modal process is not (fully) automatic; instead, it is context-contingent. More generally, the present findings provide novel insights into the importance of context-related factors in controlling information processing across the senses, and call for a revision of current models of automaticity in cognitive sciences.

Brain circuit–gene expression relationships and neuroplasticity of multisensory cortices in blind children

“…Indeed, we describe similar connectivity increases but at the indirect connectivity level, meaning that a significant amount of cross-modal interactions between visual cortex and other primary systems is likely determined by multimodal relay stations. Furthermore, we theorize that functional connectivity decreases between the occipital lobe and other primary systems in blind adults (14,15) may relate to impaired connections between primary visual areas and the multimodal integration network and that-potentially as an adaptive phenomenon-increased connectivity between somatosensory or auditory cortices and multimodal areas develop (both phenomena seen in our findings).…”

“…Historically, the visual cortex was considered to be exclusively devoted to visual processes, but there is now evidence that activity within the human visual cortex plays an active role in multisensory processes (14,15). In recent years, the occipital cortex, as well as adjacent ventral areas such as the fusiform gyrus (16,17), have been the main regions of interest in the study of cross-modal phenomena in blindness.…”

Behavioral and Neural Foundations of Multisensory Face-Voice Perception in Infancy

“…In adults, neural attenuation for bimodal synchrony is thought to reflect more efficient processing and has been documented in both early sensory and later attentional responses (Belse et al, 2004;Pilling, 2009;van Wassenhove, Grant, & Poepple, 2005), whereas attenuation has only been observed in later going attentional responses in infants (Hyde et al, 2011, Grossmann et al, 2006Reynolds et al, 2014;Vogel et al, 2012). Inconsistencies in the functional brain response between different experimental contexts in infants should not be taken as a limitation of the work, as most current views of multimodal processing and intersensory perception in adults conclude that context has a large effect on how, when, and where the brain is engaged (e.g., van Atteveldt et al, 2014;Matusz, Retsa, & Murray, 2016;Murray, Lewkowicz, Amedi, & Wallace, 2016). However, the fact that neural attenuation for bimodal synchrony has been documented exclusively in later attentional processing in infants but often appears during earlier sensory processing in adults may reflect a genuine developmental difference between infants and adults in efficiency and time course.…”

A multisensory perspective on object memory

“…Nonetheless and first, it is now well established that visual areas such as the LOC and auditory areas such as the STC demonstrate multisensory convergence and integration (see, e.g., Matusz, Retsa, & Murray, 2016; Sarmiento, Matusz, Sanabria, & Murray, 2015; reviewed in Doehrmann & Naumer, 2008; Murray et al, 2016b; ten Oever et al, 2016). Second, microelectrode recordings in monkey posterior infero-temporal (IT) cortex, for which the LOC is considered to be the human homologue, as well as visual area V4, demonstrate selective delay-period responses on a delayed match-to-sample task for specific multisensory and unisensory pairings (e.g., Colombo & Gross, 1994; Gibson & Maunsell, 1997; Haenny, Maunsell, & Schiller, 1988; Maunsell, Sclar, Nealey, & DePriest, 1991; see also Goulet & Murray, 2001).…”