The role of conflict processing in multisensory perception: behavioural and electroencephalography evidence

Abstract

To form coherent multisensory perceptual representations, the brain must solve a causal inference problem: to decide if two sensory cues originated from the same event and should be combined, or if they came from different events and should be processed independently. According to current models of multisensory integration, during this process, the integrated (common cause) and segregated (different causes) internal perceptual models are entertained. In the present study, we propose that the causal inference process involves competition between these alternative perceptual models that engages the brain mechanisms of conflict processing. To test this hypothesis, we conducted two experiments, measuring reaction times (RTs) and electroencephalography, using an audiovisual ventriloquist illusion paradigm with varying degrees of intersensory disparities. Consistent with our hypotheses, incongruent trials led to slower RTs and higher fronto-medial theta power, both indicative of conflict. We also predicted that intermediate disparities would yield slower RTs and higher theta power when compared to congruent stimuli and to large disparities, owing to the steeper competition between causal models. Although this prediction was only validated in the RT study, both experiments displayed the anticipated trend. In conclusion, our findings suggest a potential involvement of the conflict mechanisms in multisensory integration of spatial information. This article is part of the theme issue ‘Decision and control processes in multisensory perception’.

Perceptual oddities: assessing the relationship between film editing and prediction processes

“…In electroencephalography (EEG), an increase in oscillatory power in the theta band (5-7 Hz) in mid-frontal areas distributed over the ACC, has been proposed as the lingua franca reflecting a common adaptation mechanism in a variety of situations involving ambiguity ( prediction errors, input conflict, post-error adjustments, surprise and novelty) [65][66][67][68][69][70]. This oscillatory correlate has since been widely employed in conflict tasks at varying levels of information processing [27][28][29][30]71], and we adopt it in what follows as a neural signature of prediction errors between anticipated and actual sensory input at edit boundaries in cinema.…”

“…Stroop or Flanker tasks). 1 However, the original formulation of the framework, as well as later updates make clear that conflict, like prediction errors in the PP framework, can occur at any level of the information processing hierarchy, including perceptual representations [16,17,22,23], a claim that is corroborated by a mounting body of evidence across different domains (semantics [24,25], crossmodal and visual perception [26][27][28][29], and surrealism in art and advertisement [30]). Should the theory of conflict monitoring be applicable to sensory processing and perception at large, then it could serve as the regulatory mechanism for prediction error.…”

How the brain controls decision making in a multisensory world

“…Marly et al. [ 30 ] explore whether this arbitration between sensory integration and segregation involves conflict monitoring and cognitive control. In support of this conjecture, the researchers observed longer response times and greater theta oscillatory power at intermediate audiovisual spatial disparities when observers are least certain about whether signals come from common or independent sources.…”

Conflict regulation in flavour perception

“…Conflict mechanisms are increasingly thought to play a critical role in shaping complex perceptual experiences despite potentially ambiguous sensory signals. Evidence that conflict mechanisms regulate disparities in sensory information in favour of coherent processing comes from unimodal (visual) conflict during binocular rivalry (Drew et al, 2022), crossmodal conflict during the ventriloquist illusion (Marly et al, 2023) or the McGurk illusion (Morís Fernández et al, 2017 and conflict in more complex scenarios, such as in viewing surrealist art and advertisement (Ruzzoli et al, 2020). In each of these perceptual scenarios, conflict mechanisms call for increased attention towards selectively stifling or magnifying sensory ambiguity to make sense of the current sensory information.…”