Multisensory decisions from self to world

Abstract

Classic Bayesian models of perceptual inference describe how an ideal observer would integrate ‘unisensory’ measurements (multisensory integration) and attribute sensory signals to their origin(s) (causal inference). However, in the brain, sensory signals are always received in the context of a multisensory bodily state—namely, in combination with other senses. Moreover, sensory signals from both interoceptive sensing of one's own body and exteroceptive sensing of the world are highly interdependent and never occur in isolation. Thus, the observer must fundamentally determine whether each sensory observation is from an external (versus internal, self-generated) source to even be considered for integration. Critically, solving this primary causal inference problem requires knowledge of multisensory and sensorimotor dependencies. Thus, multisensory processing is needed to separate sensory signals. These multisensory processes enable us to simultaneously form a sense of self and form distinct perceptual decisions about the external world. In this opinion paper, we review and discuss the similarities and distinctions between multisensory decisions underlying the sense of self and those directed at acquiring information about the world. We call attention to the fact that heterogeneous multisensory processes take place all along the neural hierarchy (even in forming ‘unisensory’ observations) and argue that more integration of these aspects, in theory and experiment, is required to obtain a more comprehensive understanding of multisensory brain function. This article is part of the theme issue ‘Decision and control processes in multisensory perception’.

The Body Knows Better: Sensorimotor signals reveal "Suboptimal" inference of the Sense of Agency in the human mind

“…Indeed, participants show wide "tuning curves" for the sense of agency accepting actions with delays of up to 120 ms as their own (Krugwasser et al, 2022(Krugwasser et al, , 2019Stern et al, 2021Stern et al, , 2020. Thus, while task performance for sensorimotor conflicts is suboptimal, SoA and the bodily self are preserved in the face of noisy environments (Zaidel and Salomon, 2023).…”

Rats rely on airflow cues for self-motion perception

“…Multisensory processing is a broad and rich field of investigation, which comprises many functions besides multisensory integration. These include multisensory recalibration (Zaidel et al, 2011(Zaidel et al, , 2013(Zaidel et al, , 2021Zeng et al, 2022;Shalom-Sperber et al, 2022), causal inference (Kording et al, 2007;Acerbi et al, 2018;Dokka et al, 2019) and even perception of one's self (Zaidel & Salomon, 2023). Moreover, self-motion perception, and other multisensory functions, are often altered in brain disorders such as Parkinson's disease (Halperin et al, 2020;Yakubovich et al, 2020) and autism (Zaidel et al, 2015;Feigin et al, 2021;Noel et al, 2022).…”

The bodily self from psychosis to psychedelics

“…For example, if the visual outcome of an action is modified temporally e.g., 76 – 78 , anatomically e.g., 56 , or spatially e.g., 79 – 81 , participants do not experience authorship of the action. Thus, both Body Ownership and Sense of Agency rely on integration of sensory signals, however, Sense of Agency utilizes information from volitional action which can be used to improve inferences on the bodily self 82 .…”