The multisensory function of the human primary visual cortex

Abstract

2It has been nearly ten years since Ghazanfar and Schroeder (2006) proposed that the neocortex is essentially multisensory in nature. However, it is only recently that sufficient and hard evidence that supports this proposal has accrued. We review evidence that activity within the human primary visual cortex plays an active role in multisensory processes and directly impacts behavioural outcome. This evidence emerges from a full pallet of human brain imaging and brain mapping methods with which multisensory processes are quantitatively assessed by taking advantage of particular strengths of each technique as well as advances in signal analyses. Several general conclusions about multisensory processes in primary visual cortex of humans are supported relatively solidly. First, haemodynamic methods (fMRI/PET) show that there is both convergence and integration occurring within primary visual cortex. Second, primary visual cortex is involved in multisensory processes during early post-stimulus stages (as revealed by EEG/ERP/ERFs as well as TMS). Third, multisensory effects in primary visual cortex directly impact behaviour and perception, as revealed by correlational (EEG/ERPs/ERFs) as well as more causal measures (TMS/tACS). While the provocative claim of Ghazanfar and Schroeder (2006) that the whole of neocortex is multisensory in function has yet to be demonstrated, this can now be considered established in the case of the human primary visual cortex.

The context-contingent nature of cross-modal activations of the visual cortex

“…Accordingly, sounds can modulate activity within visual cortices within the first 100ms post-stimulus onset, can directly impact behaviour, and can do so outside the listener's conscious awareness (Romei et al, 2007(Romei et al, , 2009Spierer et al, 2013;reviewed in Murray et al, 2015a). In line with this general notion, the multisensory processing that occurs within the first 100ms post-stimulus is increasingly revealed to do so largely independently (at least in its presence) of the top-down control of goals, semantics, or stimulus context (the eMSI; De Meo et al 2015; see also Murray et al, 2015a;Ten Oever et al, in revisions). In this regard, the timing of the onset of the ACOP at ~250-300ms is considerably later than the timing of other cross-modal and multisensory processes that could be considered automatic inasmuch as they occur independently of a multitude of top-down influences.…”

“…Importantly, each of these processes is subject to a differing degree to constraints imposed by the current behavioural goals of the observer, which will determine the efficacy of a particular cross-modal influence. While at least some multisensory processes, such as those based on the detection of multisensory simultaneity, occur independently of the task-relevance of the other-modality signals (Matusz et al, 2011;De Meo et al, 2015;Murray et al, 2015a;Ten Oever et al, in revisions), orienting of involuntary spatial attention might be less impervious to it.…”

A multisensory perspective on object memory

“…Indeed, under such multisensory circumstances, profound changes in behaviour and perception can be elicited, and these changes are accompanied by striking changes in the patterns of brain activation and the networks that are engaged. Auditory-visual multisensory processes have been identified throughout functional cortical hierarchies, including primary cortices (reviewed in Murray et al, 2016a) infero-temporal and superior temporal regions (reviewed in Lewis 2010 for the case of auditory-visual object processing) as well as prefrontal regions (reviewed in Murray and Wallace, 2012). Although much emphasis has been placed on behavioural and perceptual processes, recent work has also shown that the presentation of sensory stimuli in a multisensory manner can also have profound effects on our memories, and provide important clues as to why you can recognize Sarah better than her friends on your second meeting in the example provided above.…”

Sensory Augmentation through Tissue Conduction

“…The ubiquity of multisensory neurons (that can receive inputs from two or more sensory domains) in the brain indicates that multisensory integration is not limited to 'higher' cognitive processes but can occur at more fundamental levels. For instance, neurons in the primary visual cortex receive inputs from the primary auditory cortex [20]. Multimodality can be discussed in terms of cross-modal effects (where stimulus input to one modality alters the perceptual conclusions in another), for examples, the motion bounce illusion, see [21] and the McGurk effect [22].…”