Using the past to estimate sensory uncertainty

Abstract

To form the most reliable percept of the environment, the brain needs to represent sensory uncertainty. Current theories of perceptual inference assume that the brain computes sensory uncertainty instantaneously and independently for each stimulus.In a series of psychophysics experiments human observers localized auditory signals that were presented in synchrony with spatially disparate visual signals. Critically, the visual noise changed dynamically over time with or without intermittent jumps. Our results show that observers integrate audiovisual inputs weighted by sensory reliability estimates that combine information from past and current signals as predicted by an optimal Bayesian learner or approximate strategies of exponential discountingOur results challenge classical models of perceptual inference where sensory uncertainty estimates depend only on the current stimulus. They demonstrate that the brain capitalizes on the temporal dynamics of the external world and estimates sensory uncertainty by combining past experiences with new incoming sensory signals.

The neurophysiological basis of short- and long-term ventriloquism aftereffects

“…This regularity of the sensory environment may hence involve frontal regions that exploit this regularity on longer times beyond the more immediate aftereffect arising from parietal regions. Such a divided role of parietal and frontal regions in contributing to multisensory recalibration is in line with the notion that parietal regions contribute to the fusion of multisensory information within a trial, while frontal regions are also engaged in the causal inference of whether different stimuli likely arise from a common source (Cao et al 2019;Rohe et al 2019;Rohe and Noppeney 2015).Such an inference process benefits from knowledge about the recent stimulus history (Beierholm et al 2019). It remains to be understood whether the same or distinct frontal regions contribute to causal inference within a trial and the fostering of recalibration based on the cumulative stimulus history.…”

The Neurophysiological Basis of the Trial-Wise and Cumulative Ventriloquism Aftereffects

“…One possibility is that parietal regions guide the aftereffect based on the more immediate stimulus history, while frontal processes exploit the regularity over longer time scales. Such a divided role fits with the notion that parietal regions contribute to the immediate fusion of multisensory information within a trial, while frontal regions help differentiating whether two stimuli arise from a common source (Cao et al, 2019;Rohe et al, 2019;Rohe & Noppeney, 2015), a process known to benefit from knowledge about stimulus history (Beierholm et al, 2019). Future work should investigate whether the same or distinct frontal regions contribute to causal inference within a trial and the fostering of recalibration based on the cumulative stimulus history.…”

Retinotopic organization of feedback projections in primate early visual cortex: implications for active vision

“…In recent years advances in signal analysis have led to a revision of the notion of unimodal primary areas suggesting that the neocortex might essentially be multisensory in function (Cappe and Barone, 2005; Fu et al, 2003; Gau et al, 2020; Ghazanfar and Schroeder, 2006; Hasson et al, 2016; Henschke et al, 2015; Liang et al, 2013; Macaluso and Driver, 2005; Murray et al, 2016). The major focus on the projection of non-visual areas to early visual cortex in the primate brain has been on those originating from auditory cortex which in addition to multisensory integration and their impact on perception (Bedny, 2017; Deen et al, 2015; Kanjlia et al, 2021; Kupers and Ptito, 2014; van Wassenhove et al, 2005), have been implicated in cortical plasticity and cross modal reorganization (Gau et al, 2020; Vetter et al, 2020) and as a model system for investigating predictive processing in the cortex (Beierholm et al, 2020; Gau et al, 2020; Rohe et al, 2019; Rohe and Noppeney, 2015; Vetter et al, 2020).…”