Listen to Your Heart: Examining Modality Dominance Using Cross-Modal Oddball Tasks

Abstract

The current study used cross-modal oddball tasks to examine cardiac and behavioral responses to changing auditory and visual information. When instructed to press the same button for auditory and visual oddballs, auditory dominance was found with cross-modal presentation slowing down visual response times more than auditory response times (Experiment 1). When instructed to make separate responses to auditory and visual oddballs, visual dominance was found with cross-modal presentation decreasing auditory discrimination, and participants also made more visual-based than auditory-based errors on cross-modal trials (Experiment 2). Experiment 3 increased task demands while requiring a single button press and found evidence of auditory dominance, suggesting that it is unlikely that increased task demands can account for the reversal in Experiment 2. Auditory processing speed was the best predictor of auditory dominance, with auditory dominance being stronger in participants who were slower at processing the sounds, whereas auditory and visual processing speed and baseline heart rate variability did not predict visual dominance. Examination of cardiac responses that were time-locked with stimulus onset showed cross-modal facilitation effects, with auditory and visual discrimination occurring earlier in the course of processing in the cross-modal condition than in the unimodal conditions. The current findings showing that response demand manipulations reversed modality dominance and that time-locked cardiac responses show cross-modal facilitation, not interference, suggest that auditory and visual dominance effects may both be occurring later in the course of processing, not from disrupted encoding.

Increases in pre-stimulus theta and alpha oscillations precede successful encoding of crossmodal associations

“…The present results are only partially in line with the previous evidence, as decreases in theta power associated with better memory performance were also found in the lower frequency of the theta band. One possible explanation could be that the power in REM trials decreased further due to longer processing time of the auditory stimuli of the pairs 61 . This decrease could then be interpreted as a positive effect for behavior.…”

Stimulus temporal uncertainty balances intersensory dominance

“…In these studies, vision is typically dominant over audition or touch when it comes to speedily discriminating the modalities of targets presented at an expected time and space (for reviews, see Hirst et al, 2018;Spence et al, 2012). That said, the Colavita visual dominance effect is susceptible to cognitive controls (Koppen & Spence, 2007b;Shapiro et al, 1984;Sinnett et al, 2007) and working memory (Ngo et al, 2011;Robinson et al, 2016Robinson et al, , 2018Robinson et al, , 2020. Here, a bottom-up stimulus factor was demonstrated, where, in the classic simple-discrimination paradigm, increasing stimulus temporal uncertainty was found to effectively balance the dominance between vision and audition.…”

“…However, in a meta-analysis study, Hirst et al (2018) reported that there was no significant pattern of sensory dominance in children. 2 Several studies that reported an auditory dominance effect mainly used experimental paradigms involving working memory, such as two-back repetition detection (Ngo et al, 2011), oddball detection (Robinson et al, 2016(Robinson et al, , 2020, and change detection (Dunifon et al, 2016;Robinson et al, 2018), rather than a simple discrimination task that is commonly used to demonstrate the Colavita visual dominance effect (Colavita, 1974;Hirst et al, 2018;Spence et al, 2012). In addition, Robinson and colleagues often observed an auditory dominance effect in terms of response time measures (Robinson et al, 2016(Robinson et al, , 2018.…”

Spectral signatures of cross-modal attentional control in the adolescent brain and their link with physical activity and aerobic fitness levels

“…While each sensory cortex is designed to process a specific type of information, there seems to be considerable overlap in their representations between them (Karim et al, 2021;Rapp & Hendel, 2003). There is growing evidence suggesting that the processing of stimuli in one modality can be enhanced (Dematte et al, 2006;Zhao et al, 2021) or diminished (Geangu et al, 2021;Robinson et al, 2020) by the simultaneous presentation of stimuli in different modalities. Therefore, besides the well-established cortical areas that are suggested to provide "central attentional control" (i.e., prefrontal areas, cingulate gyrus, and posterior parietal cortex [Lepsien & Nobre, 2006]), it is possible that attentional demands in cross-modal tasks also modulate the sensory processing of information (Lage-Castellanos et al, 2022;van Atteveldt et al, 2014).…”