Presentation Probability of Visual–Auditory Pairs Modulates Visually Induced Auditory Predictions

Abstract

Predictions about forthcoming auditory events can be established on the basis of preceding visual information. Sounds being incongruent to predictive visual information have been found to elicit an enhanced negative ERP in the latency range of the auditory N1 compared with physically identical sounds being preceded by congruent visual information. This so-called incongruency response (IR) is interpreted as reduced prediction error for predicted sounds at a sensory level. The main purpose of this study was to examine the impact of probability manipulations on the IR. We manipulated the probability with which particular congruent visual–auditory pairs were presented (83/17 vs. 50/50 condition). This manipulation led to two conditions with different strengths of the association of visual with auditory information. A visual cue was presented either above or below a fixation cross and was followed by either a high- or low-pitched sound. In 90% of trials, the visual cue correctly predicted the subsequent sound. In one condition, one of the sounds was presented more frequently (83% of trials), whereas in the other condition both sounds were presented with equal probability (50% of trials). Therefore, in the 83/17 condition, one congruent combination of visual cue and corresponding sound was presented more frequently than the other combinations presumably leading to a stronger visual–auditory association. A significant IR for unpredicted compared with predicted but otherwise identical sounds was observed only in the 83/17 condition, but not in the 50/50 condition, where both congruent visual cue–sound combinations were presented with equal probability. We also tested whether the processing of the prediction violation is dependent on the task relevance of the visual information. Therefore, we contrasted a visual–auditory matching task with a pitch discrimination task. It turned out that the task only had an impact on the behavioral performance but not on the prediction error signals. Results suggest that the generation of visual-to-auditory sensory predictions is facilitated by a strong association between the visual cue and the predicted sound (83/17 condition) but is not influenced by the task relevance of the visual information.

Cross‐modal predictive processing depends on context rather than local contingencies

“…In the latter condition, the probability of, for instance, the high visual-cue-sound combination was increased relative to the low visual-cue-sound combination, leading to an overall presentation frequency of 83% high versus 17% low sounds. Stuckenberg et al (2019) only observed an IR in this specific condition, when one visual-cue-sound combination was presented more frequently than the other (83/17 condition), whereas no IR was observed when high and low visual-cuesound combinations were presented with equal probability (50/50 condition). They conclude that the increased global probability of one visual-cue-sound combination is a prerequisite for the formation of a strong bimodal association.…”

“…Given these contradicting findings (Stuckenberg et al, 2019vs. Widmann et al, 2004 and the literature on short-latency components (Wacongne et al, 2011), the present study aims to better understand the role of local repetition on the elicitation of the IR.…”

“…It has been shown that there is a cross-modal correspondence between high-pitched sounds and small, bright objects that are located high up in space (Spence, 2011). But the 50/50 condition of Stuckenberg et al (2019) revealed that this correspondence alone is not sufficient to trigger visual-based predictions. However, the change in global probability (from 50/50 to 83/17) is inherently accompanied by a change in local repetitions of one pair, as increasing the frequency of one visual-cue-sound combination over the other affects the local probability of that combination to occur.…”

“…In the context of these findings, the IR can also be regarded as an early response, which would suggest the IR to be driven by local effects. One study that might support this was conducted by Stuckenberg et al (2019). They presented an eighth note symbol either above or below a fixation cross which was succeeded by the presentation of either a high-or low-pitched sound.…”

What exactly is missing here? The sensory processing of unpredictable omissions is modulated by the specificity of expected action‐effects

“…The Bayes factor (BF 10 ) was calculated using 10.000 Monte-Carlo sampling iterations; the null hypothesis corresponded to a standardized effect size δ = 0, while the alternative hypothesis was defined as a Cauchy prior distribution centred around 0 with a scaling factor of r = 0.707 (Rouder, Morey, Speckman, & Province, 2012). In line with the Bayes factor interpretation (Jeffreys, 1961;Lee & Wagenmakers, 2013) and with previous studies reporting Bayes factors (Korka et al, 2019;Marzecová et al, 2018;Stuckenberg, Schröger, & Widmann, 2019), data were taken as moderate evidence for the alternative (or null) hypothesis if the BF 10 was greater than 3 (or lower than 0.33), while values close to 1 were considered only weakly informative. Values greater than 10 (or smaller than 0.1) were considered strong evidence for the alternative (or null) hypothesis.…”