Multisensory causal inference is feature-specific, not object-based

Badde, Stephanie; Landy, Michael S.; Adams, Wendy J.

doi:10.1098/rstb.2022.0345

Cited by 3 publications

(4 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many paradigms probe multisensory binding using paradigms involving one stimulus per modality. Whether and how results obtained using these generalize to real-life multisensory object perception remains unclear (Badde et al, 2023;Bizley, Maddox, & Lee, 2016;Duarte et al, 2023;Kirsch & Kunde, 2023;Newell et al, 2023;Tovar et al, 2020). In a step towards this direction, we asked how the presence of two visual stimuli shapes multisensory perception in the audio-visual spatial ventriloquism paradigm.…”

Section: Discussionmentioning

confidence: 99%

“…Although the task‐relevant stimulus was cued only post‐stimulus presentation, trials requiring an auditory judgement were generally more likely, and hence, we cannot rule out such an interpretation. Separating out the effects of attention and task difficulty during multisensory object perception remains one of the challenges for future work (Badde et al, 2023; Bizley, Maddox, & Lee, 2016; Noppeney, 2021).…”

Section: Discussionmentioning

confidence: 99%

“…While it is known that multisensory binding is generally shaped by the spatio-temporal relation of the stimuli, their semantic attributes, their task relevance and the attentional focus (Noppeney, 2021;Shams & Beierholm, 2022;Talsma et al, 2010), still little is known about how multiple signals from one modality are combined with one signal from another. During everyday life, we usually perceive the world as consisting of various objects, and it still remains unclear how multiple signals are bound or segregated within and across modalities to facilitate a coherent interpretation of our environment (Badde et al, 2023;Duarte et al, 2023;Newell et al, 2023;Tovar et al, 2020).…”

mentioning

confidence: 99%

See 2 more Smart Citations

Both stimulus‐specific and configurational features of multiple visual stimuli shape the spatial ventriloquism effect

Kayser,

Debats,

Heuer

2024

Eur J of Neuroscience

View full text Add to dashboard Cite

Studies on multisensory perception often focus on simplistic conditions in which one single stimulus is presented per modality. Yet, in everyday life, we usually encounter multiple signals per modality. To understand how multiple signals within and across the senses are combined, we extended the classical audio‐visual spatial ventriloquism paradigm to combine two visual stimuli with one sound. The individual visual stimuli presented in the same trial differed in their relative timing and spatial offsets to the sound, allowing us to contrast their individual and combined influence on sound localization judgements. We find that the ventriloquism bias is not dominated by a single visual stimulus but rather is shaped by the collective multisensory evidence. In particular, the contribution of an individual visual stimulus to the ventriloquism bias depends not only on its own relative spatio‐temporal alignment to the sound but also the spatio‐temporal alignment of the other visual stimulus. We propose that this pattern of multi‐stimulus multisensory integration reflects the evolution of evidence for sensory causal relations during individual trials, calling for the need to extend established models of multisensory causal inference to more naturalistic conditions. Our data also suggest that this pattern of multisensory interactions extends to the ventriloquism aftereffect, a bias in sound localization observed in unisensory judgements following a multisensory stimulus.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Both stimulus‐specific and configurational features of multiple visual stimuli shape the spatial ventriloquism effect

Kayser,

Debats,

Heuer

2024

Eur J of Neuroscience

View full text Add to dashboard Cite

show abstract

“…Badde et al. [ 31 ] examined causal inference in more complex visuo-haptic settings that incorporate two object features, slant and roughness. The degree to which visuo-haptic roughness and slant features were integrated optimally varied across participants in an uncorrelated manner, suggesting that causal inference is performed at the feature- rather than object-level.…”

Section: Overview Of the Theme Issuementioning

confidence: 99%

How the brain controls decision making in a multisensory world

Fetsch,

Noppeney

2023

Phil. Trans. R. Soc. B

View full text Add to dashboard Cite

Sensory systems evolved to provide the organism with information about the environment to guide adaptive behaviour. Neuroscientists and psychologists have traditionally considered each sense independently, a legacy of Aristotle and a natural consequence of their distinct physical and anatomical bases. However, from the point of view of the organism, perception and sensorimotor behaviour are fundamentally multi-modal; after all, each modality provides complementary information about the same world. Classic studies revealed much about where and how sensory signals are combined to improve performance, but these tended to treat multisensory integration as a static, passive, bottom-up process. It has become increasingly clear how this approach falls short, ignoring the interplay between perception and action, the temporal dynamics of the decision process and the many ways by which the brain can exert top-down control of integration. The goal of this issue is to highlight recent advances on these higher order aspects of multisensory processing, which together constitute a mainstay of our understanding of complex, natural behaviour and its neural basis. This article is part of the theme issue ‘Decision and control processes in multisensory perception’.

show abstract

Audiovisual simultaneity windows reflect temporal sensory uncertainty

Cary,

Lahdesmaki,

Badde

2024

Psychon Bull Rev

View full text Add to dashboard Cite

The ability to judge the temporal alignment of visual and auditory information is a prerequisite for multisensory integration and segregation. However, each temporal measurement is subject to error. Thus, when judging whether a visual and auditory stimulus were presented simultaneously, observers must rely on a subjective decision boundary to distinguish between measurement error and truly misaligned audiovisual signals. Here, we tested whether these decision boundaries are relaxed with increasing temporal sensory uncertainty, i.e., whether participants make the same type of adjustment an ideal observer would make. Participants judged the simultaneity of audiovisual stimulus pairs with varying temporal offset, while being immersed in different virtual environments. To obtain estimates of participants’ temporal sensory uncertainty and simultaneity criteria in each environment, an independent-channels model was fitted to their simultaneity judgments. In two experiments, participants’ simultaneity decision boundaries were predicted by their temporal uncertainty, which varied unsystematically with the environment. Hence, observers used a flexibly updated estimate of their own audiovisual temporal uncertainty to establish subjective criteria of simultaneity. This finding implies that, under typical circumstances, audiovisual simultaneity windows reflect an observer’s cross-modal temporal uncertainty.

show abstract

Multisensory causal inference is feature-specific, not object-based

Cited by 3 publications

References 34 publications

Both stimulus‐specific and configurational features of multiple visual stimuli shape the spatial ventriloquism effect

Both stimulus‐specific and configurational features of multiple visual stimuli shape the spatial ventriloquism effect

How the brain controls decision making in a multisensory world

Audiovisual simultaneity windows reflect temporal sensory uncertainty

Contact Info

Product

Resources

About