Repeated exposure to either consistently spatiotemporally congruent or consistently incongruent audiovisual stimuli modulates the audiovisual common-cause prior

Abstract

To estimate an environmental property such as object location from multiple sensory signals, the brain must infer their causal relationship. Only information originating from the same source should be integrated. This inference relies on the characteristics of the measurements, the information the sensory modalities provide on a given trial, as well as on a cross-modal common-cause prior: accumulated knowledge about the probability that cross-modal measurements originate from the same source. We examined the plasticity of this cross-modal common-cause prior. In a learning phase, participants were exposed to a series of audiovisual stimuli that were either consistently spatiotemporally congruent or consistently incongruent; participants’ audiovisual spatial integration was measured before and after this exposure. We fitted several Bayesian causal-inference models to the data; the models differed in the plasticity of the common-source prior. Model comparison revealed that, for the majority of the participants, the common-cause prior changed during the learning phase. Our findings reveal that short periods of exposure to audiovisual stimuli with a consistent causal relationship can modify the common-cause prior. In accordance with previous studies, both exposure conditions could either strengthen or weaken the common-cause prior at the participant level. Simulations imply that the direction of the prior-update might be mediated by the degree of sensory noise, the variability of the measurements of the same signal across trials, during the learning phase.

Multisensory causal inference is feature-specific, not object-based

“…The model assumes that observers establish two intermediate estimates of the to-be-judged feature, one based on optimal cue integration of visual and haptic sensory signals, and one based on their favourite modality, the modality they would choose if visual and haptic signals were from different sources. Analogous to previous implementations of Bayesian multisensory causal inference [5,19,20,31], these two intermediate estimates are averaged, weighted by the posterior probability of a common cause. Thus, if the inferred probability that the signals share a common cause is 1, the observer fully bases their perceptual decisions on the integrated estimate and the variance across visual–haptic trials is identical to that predicted by optimal cue integration (the denominator of the integration index).…”

“…We further assumed that the sensory signals, also called measurements, mv,i,mh,i were corrupted by Gaussian-distributed noise with variance σv2 and σh2. We additionally allowed the sensory signals to be biased [19,20,31], as modality-specific biases in the sensory signals are a root cause of reduced cross-modal integration effects [21].…”

“…This posterior is the product of the a priori probability that the observer assigns to the common-cause scenario and the likelihood that the sensory signals share a common cause. The common-cause prior is a top–down influence; it varies across stimuli [17], with the observer's previous experiences [18,19] as well as their attentional state [20]. In turn, the likelihood of a common cause is driven by sensory information about the stimuli.…”

Near-optimal integration of the magnitude Information of time and numerosity

“…Furthermore, the differences between the results of our two experiments suggest that the coincidence of the timing of the acquisition of each piece of information is crucial for optimal magnitude integration. Interestingly, spatialtemporal discrepancies in the acquisition of information have been shown to affect optimal integration, even in the multisensory integration of single physical quantities such as spatial location and orientation (35)(36)(37). For example, Plaisier et al (37) reported that, in the visual-haptic integration of surface orientation, optimal integration breaks down when there is a discrepancy between the visual and haptic exploration modes (i.e., the instantaneous perception of the surface orientation by touching/seeing two spots vs. sequential perception by tracing the surface).…”

Multisensory integration in the mammalian brain: diversity and flexibility in health and disease

“…(b) Spatiotemporal modulation of MSI by experience and learning during the task. Left, spatial information of each modality before and after experience or training [158][159][160][161]. Right, the proportion of simultaneity reports before and after the training at different stimulus-onset asynchronies (SOAs) [42,162,163].…”