Towards understanding how we pay attention in naturalistic visual search settings

Abstract

Traditional research on attentional control has largely focused on single senses and the importance of one s behavioural goals in controlling attentional selection, thus limiting its generalizability to real-world contexts. These contexts are both inherently multisensory and contain regularities that also contribute to attentional control. To get a better understanding of how attention is controlled in the real world, we investigated how visual attentional capture was impacted by top-down goals (indexed by task-set contingent attentional capture) and the multisensory nature of stimuli, as well as top-down contextual factors such as semantic relationships and temporal predictability of stimulus onset. Participants performed a multisensory version of Folk et al. (1992) spatial cueing paradigm, while their 129-channel event-related potentials (ERPs) were recorded. Reaction-time spatial cueing served as a behavioural measure of attentional control, while the N2pc ERP component was analysed both canonically and using a multivariate electrical neuroimaging (EN) framework. Behaviourally, target-congruent colour distractors captured attention more strongly when they were simultaneous than semantically congruent (nontarget-congruent colour distractors failed to capture attention), with no behavioural evidence for context modulating multisensory enhancements of capture. However, our EN analyses revealed context-based influences on attention to both visual and multisensory distractors, on how strongly they activated brain networks and in the type of activated brain networks. In both cases, these context-driven brain response modulations occurred early on (long before the traditional N2pc time-window), with network-based modulations at app. 30ms post-distractor, followed by strength-based modulations at app. 100ms post-distractor. Our findings revealed that in naturalistic settings, meaning, next to predictions (spatial, temporal etc.) might be a second important source of contextual information utilised to facilitate goal-directed attention. Therein, attentional selection is controlled by an interaction of one s goals, stimulus perceptual (multisensory-driven) salience and an interaction of stimulus meaning and its predictability. Our study demonstrates how investigating more traditional, lab-studied control mechanisms and processes more typical for everyday life reveals a complex interplay between goal-, stimulus- and context-based processes in attentional control. As such, our findings call for a revision of traditional models of visual attentional control to account for the role of both contextual and multisensory control mechanisms.

The development of attentional control mechanisms in multisensory environments

“…Despite adapting the original paradigm to children and adding an EEG measurement, also adults in our study showed the behavioural MSE effect. This corroborates the particular salience of multisensory distractors ( Santangelo and Spence, 2007 ; Talsma et al, 2010 ; van der Burg et al, 2011 ; Matusz and Eimer, 2011 ; Matusz et al, 2015 , 2019a ; Turoman et al, 2020a ), further supported by evidence that multisensory integration can occur at stages of brain processing preceding those influenced by top-down processes ( Giard and Peronnet, 1999 ; Cappe et al, 2010 ; reviewed in Talsma et al, 2010 ; De Meo et al, 2015 ; Murray et al, 2016 ; ten Oever et al, 2016 ). Surprisingly, none of the children groups showed MSE in behaviour, even though children supposedly have weaker attentional control than adults ( Bunge et al, 2002 ; Hwang et al, 2010 ) and so should be theoretically more sensitive to more salient distractors.…”

“…paying attention to fast-disappearing targets of a particular colour embedded in an array of similar coloured shapes) and cross-modal audiovisual interactions within task-irrelevant objects that may be less automated in children. Finally, the behavioural MSE itself is not large even in adults, ranging between 5 and 10 ms (see also Matusz and Eimer, 2011 ; Turoman et al, 2020a ). Notwithstanding, using EN analyses we revealed the sensitivity of children’s brains to the multisensory nature of distractors and the recruitment of adult-like neuro-cognitive mechanisms for this purpose, from 8 years onwards.…”

“…Furthermore, classical N2pc analyses take into account much less EEG data than EN analyses, which suggests that EN analyses may detect effects that the canonical N2pc analyses may miss. In combination with rigorous paradigms and analyses of well-understood EEG correlates of cognitive processes, an EN approach offers a powerful tool to distinguishing between different accounts of cognitive processes, including multisensory attentional control ( Matusz et al, 2019b ; Turoman et al, 2020a ).…”

The development of attentional control mechanisms in multisensory environments

“…Despite adapting the original paradigm to children and adding an EEG measurement, also adults in our study showed the behavioural MSE effect. This corroborates the particular salience of multisensory distractors (Santangelo & Spence 2007; Talsma et al 2010; van der Burg et al 2011; Matusz & Eimer 2011; Matusz et al 2015; 2019a; Turoman et al 2020a), further supported by evidence that multisensory integration can occur at stages of brain processing preceding those influenced by top-down processes (Giard & Peronnet 1999; Cappe et al 2010; reviewed in Talsma et al 2010; De Meo et al 2015; Murray et al 2016; ten Oever et al 2016). Surprisingly, none of the children groups showed MSE in behaviour, even though children supposedly have weaker attentional control than adults (Bunge et al 2002; Hwang et al 2010) and so should be theoretically more sensitive to more salient distractors.…”

“…Furthermore, classical N2pc analyses take into account much less EEG data than EN analyses, which suggests that EN analyses may detect effects that the canonical N2pc analyses may miss. In combination with rigorous paradigms and analyses of well-understood EEG correlates of cognitive processes, an EN approach offers a powerful tool to distinguishing between different accounts of cognitive processes, including multisensory attentional control (Matusz et al 2019b; Turoman et al 2020a).…”