The neural dynamics of hierarchical Bayesian causal inference in multisensory perception

Abstract

Transforming the barrage of sensory signals into a coherent multisensory percept relies on solving the binding problem – deciding whether signals come from a common cause and should be integrated or, instead, segregated. Human observers typically arbitrate between integration and segregation consistent with Bayesian Causal Inference, but the neural mechanisms remain poorly understood. Here, we presented people with audiovisual sequences that varied in the number of flashes and beeps, then combined Bayesian modelling and EEG representational similarity analyses. Our data suggest that the brain initially represents the number of flashes and beeps independently. Later, it computes their numbers by averaging the forced-fusion and segregation estimates weighted by the probabilities of common and independent cause models (i.e. model averaging). Crucially, prestimulus oscillatory alpha power and phase correlate with observers’ prior beliefs about the world’s causal structure that guide their arbitration between sensory integration and segregation.

Alpha Activity Reflects the Magnitude of an Individual Bias in Human Perception

“…There is evidence from psychophysical experiments supporting this hypothesis in the temporal (Van der Burg, Alais, & Cass, 2015) and the spatial domain (Bruns & Röder, 2015;Bruns & Röder, 2017;Watson, Akeroyd, Roach, & S.Webb, 2019). A recent neuroimaging study showed that pre-stimulus alpha power predicted the tendency to bind audiovisual events, but played no role in mediating the influence of the recent history (Rohe, Ehlis, & Noppeney, 2019). This dissociation between prolonged and single-trial recalibration could stem from the need to strike a balance between the sustainability of an optimal processing state, and the flexibility necessary to cope with rapid changes.…”

“…One neural signature involved in this process supposedly is alpha band activity (van Kerkoerle, et al, 2014;Michalareas, et al, 2016;Sherman, Kanai, Seth, & VanRullen, 2016;Mayer, Schwiedrzik, Wibral, Singer, & Melloni, 2016), generally known to shape the perception of forthcoming stimuli (Ergenoglu, et al, 2004;Hanslmayr, et al, 2007;van Dijk, Schoffelen, Oostenveld, & Jensen, 2008;Mathewson, Gratton, Fabiani, Beck, & Ro, 2009). Recent studies have suggested that pre-stimulus alpha activity may reflect the criterion used to commit a specific response and may hence reflect a perceptual or decisional bias (Limbach & Corballis, 2016;Iemi, Chaumon, Crouzet, & Busch, 2017;Craddock, Poliakoff, El-deredy, Klepousniotou, & Lloyd, 2017;Iemi & Busch, 2018;Rohe, Ehlis, & Noppeney, 2019). Along such a role in perceptual decision making, alpha activity was shown to correlate with subjective awareness (Benwell, et al, 2017;Lange, Oostenveld, & Fries, 2013;Gulbinaite, İlhan, & VanRullen, 2017) and decision confidence (Samaha, Iemi, & Postle, 2017;Wöstmann, Waschke, & Obleser, 2018).…”

Brain dynamics for confidence-weighted learning

“…Our results suggest thus that confidence about predictions modulates intrinsic properties of the brain state to amplify or dampen surprise responses evoked by discrepant observations. Meyniel et al, 2015b), the weight of evidence (Rohe et al, 2019), the precision of predictions (Iglesias et al, 2013;Mathys et al, 2014;Vossel et al, 2014) discussed at the end of this article.Here, we propose to use optimal Bayesian models as a benchmark to formalize, at a computational level, the learning process. In particular, we formalize the notion of discrepancy between…”

“…In such models, the update is not only guided by discrepant observations, it is also regulated by confidence about predictions: for a given discrepancy, the update is smaller when the confidence associated with the prediction was larger. This confidence-weighting principle is not specific to learning, it is generally applicable whenever several sources of information must be combined (Ernst and Banks, 2002;Ma et al, 2006;Bang et al, 2014Bang et al, , 2017Meyniel et al, 2015b;Rohe et al, 2019). In a learning context, confidence should set the balance between predictions and new data.…”

Late- but not early-onset blindness impairs the development of audio-haptic multisensory integration

“…However, the results also suggest that these processes serve an adaptive purpose by allowing early sensory experience to imprint on the developing brain and preparing the developing individual for the sensory environment they are likely to experience later in life. That is, throughout the first eight years in life, the system accumulates sensory experience in order to gauge the reliability of the different sensory modalities that they will likely use later (Noppeney, Ostwald, & Werner, 2010), and to distribute modality-specific weights accordingly (Rohe, Ehlis, & Noppeney, 2019). If the early sensory environment (e.g.…”