Selective attention to visual stimuli can spread cross-modally to task-irrelevant auditory stimuli through either the stimulus-driven binding mechanism or the representation-driven priming mechanism. The stimulus-driven attentional spreading occurs whenever a task-irrelevant sound is delivered simultaneously with a spatially attended visual stimulus, whereas the representation-driven attentional spreading occurs only when the object representation of the sound is congruent with that of the to-be-attended visual object. The current study recorded event-related potentials in a space-selective visual object-recognition task to examine the exact roles of spacebased visual selective attention in both the stimulus-driven and representation-driven cross-modal attentional spreading, which remain controversial in the literature. Our results yielded that the representation-driven auditory Nd component (200-400 ms after sound onset) did not differ according to whether the peripheral visual representations of audiovisual target objects were spatially attended or not, but was decreased when the auditory representations of target objects were presented alone. In contrast, the stimulus-driven auditory Nd component (200-300 ms) was decreased but still prominent when the peripheral visual constituents of audiovisual nontarget objects were spatially unattended. These findings demonstrate not only that the representation-driven attentional spreading is independent of space-based visual selective attention and benefits in an all-or-nothing manner from object-based visual selection for actually presented visual representations of target objects, but also that although the stimulus-driven attentional spreading is modulated by space-based visual selective attention, attending to visual modality per se is more likely to be the endogenous determinant of the stimulus-driven attentional spreading.

The visual attentional blink can be substantially reduced by delivering a task‐irrelevant sound synchronously with the second visual target (T2), and this effect is further modulated by the semantic congruency between the sound and T2. However, whether the cross‐modal benefit originates from audiovisual interactions or sound‐induced alertness remains controversial, and whether the semantic congruency effect is contingent on audiovisual temporal synchrony needs further investigation. The current study investigated these questions by recording event‐related potentials (ERPs) in a visual attentional blink task wherein a sound could either synchronize with T2, precede T2 by 200 ms, be delayed by 100 ms, or be absent, and could be either semantically congruent or incongruent with T2 when delivered. The behavioral data showed that both the cross‐modal boost of T2 discrimination and the further semantic modulation were the largest when the sound synchronized with T2. In parallel, the ERP data yielded that both the early occipital cross‐modal P195 component (192–228 ms after T2 onset) and late parietal cross‐modal N440 component (424–448 ms) were prominent only when the sound synchronized with T2, with the former being elicited solely when the sound was further semantically congruent whereas the latter occurring only when that sound was incongruent. These findings demonstrate not only that the cross‐modal boost of T2 discrimination during the attentional blink stems from early audiovisual interactions and the semantic congruency effect depends on audiovisual temporal synchrony, but also that the semantic modulation can unfold at the early stage of visual discrimination processing.

No abstract