Multisensory integration in the mammalian brain: diversity and flexibility in health and disease

Abstract

Multisensory integration (MSI) occurs in a variety of brain areas, spanning cortical and subcortical regions. In traditional studies on sensory processing, the sensory cortices have been considered for processing sensory information in a modality-specific manner. The sensory cortices, however, send the information to other cortical and subcortical areas, including the higher association cortices and the other sensory cortices, where the multiple modality inputs converge and integrate to generate a meaningful percept. This integration process is neither simple nor fixed because these brain areas interact with each other via complicated circuits, which can be modulated by numerous internal and external conditions. As a result, dynamic MSI makes multisensory decisions flexible and adaptive in behaving animals. Impairments in MSI occur in many psychiatric disorders, which may result in an altered perception of the multisensory stimuli and an abnormal reaction to them. This review discusses the diversity and flexibility of MSI in mammals, including humans, primates and rodents, as well as the brain areas involved. It further explains how such flexibility influences perceptual experiences in behaving animals in both health and disease. This article is part of the theme issue ‘Decision and control processes in multisensory perception’.

The link between space and time along the human cortical hierarchy

“…Interest-ingly, preference changes were more pronounced in parietal and frontal regions [35]. Parietal cortex is the brain area where different kinds of integration processes are known to happen, for example multisensory [36] and visuo-motor [37], and where different visual magnitudes like stimulus size, duration and numerosity are processed. The intraparietal sulcus is also the place where space, time, size and numerosity maps have been found to partially overlap (for a review see [38]).…”

Low vision aids and age are associated with Müller-Lyer illusion in congenital visually impaired children

“…The studies in visually impaired humans have also suggested complex perceptual deficits in downstream extrastriate and later specialized cortical areas (Simmers and Bex, 2004;Simmers et al, 2005). The finding that children with congenital visual impairment show persistent deficits in visual integration approaching adulthood, which differentiates it from other conditions, such as schizophrenia or autism, that only delay the development of the integration rather than causing permanent deficits (Pearl et al, 2009;Stevenson et al, 2014;Choi et al, 2023). This phenomenon may be explained by the difference in the pathophysiology underlying these diverse conditions.…”

How the brain controls decision making in a multisensory world

“…Choi et al. [ 8 ] offer a comprehensive review of the anatomy and physiology of multisensory integration in mammals, illustrating how multi-modal processing is utterly widespread throughout both the cerebral cortex and subcortically, and how these complex brain networks may be disrupted in neurological and psychiatric disease. The flexibility of routing and integration in these networks should feature prominently in our best theories of perceptual decision making and could be the key to unlocking new therapeutic strategies as well.…”