Extra-foveal Processing of Object Semantics Guides Early Overt Attention During Visual Search

Abstract

Eye-tracking studies using arrays of objects have demonstrated that some high-level processing of object semantics can occur in extra-foveal vision, but its role on the allocation of early overt attention is still unclear. This eye-tracking visual search study contributes novel findings by examining the role of object-to-object semantic relatedness and visual saliency on search responses and eye-movement behaviour across arrays of increasing size (3, 5, 7). Our data show that a critical object was looked at earlier and for longer when it was semantically unrelated than related to the other objects in the display, both when it was the search target (target-present trials) and when it was a target’s semantically related competitor (target-absent trials). Semantic relatedness effects manifested already during the very first fixation after array onset, were consistently found for increasing set sizes, and were independent of low-level visual saliency, which did not play any role. We conclude that object semantics can be extracted early in extra-foveal vision and capture overt attention from the very first fixation. These findings pose a challenge to models of visual attention which assume that overt attention is guided by the visual appearance of stimuli, rather than by their semantics.

Preserved Extra-Foveal Processing of Object Semantics in Alzheimer’s Disease

“…This manipulation critically differs from Seckin et al, (2016), where distractors belonging to different semantic categories were presented together with semantically related distractors. We previously showed that in young adults, our manipulation led to a "semantic pop-out" effect whereby participants looked at a critical object earlier, and for longer, when it was semantically unrelated than related to the other distractors (Cimminella et al, 2020). This finding indicated that, in young adults, the rapid and complex extraction of object semantics occurs in extra-foveal vision, and it influences the early allocation of overt attention (see also Nuthmann et al, 2019;and Borges et al, 2020, for corroborating results in healthy older adults using naturalistic scenes).…”

“…Therefore, the aims of the present study were twofold: (1) to establish whether processing of semantic information about objects is preserved in AD and, if so, (2) to determine whether it can occur through extra-foveal vision, as shown in our previous work on healthy younger adults (Cimminella, et al 2020). We used the experimental paradigm we previously developed, reported in observed ceiling search performance on young adults in our previous study, but for older adults, a semantically related search object (e.g., a car) in target-absent trials may lead to greater memory demands, i.e., the object is not present in the context.…”

“…We used the 128 five-object arrays from Cimminella et al (2020) and created a further 32 arrays (i.e., a total of 160 items) by crossing the visual saliency (non-salient, salient) and the semantic relatedness (unrelated, related) of 40 critical objects (i.e., 40 * 4); refer to Supplementary Material A for miniatures of the experimental arrays. The position of the critical object in the array was rotated across items to prevent location biases.…”

“…A total of 200 objects were taken from the Bank of Standardized Stimuli (BOSS) database (Brodeur, Dionne-Dostie, Montreuil, & Lepage, 2010;Brodeur, Guérard, & Bouras, 2014), and used to create the experimental arrays (i.e., 40 as critical objects and 160 as distractors). All objects were univocally classifiable into 20 semantic categories (see Cimminella et al, 2020, for details on the semantic category norms). In addition, all objects were named by a sample of Italian participants to obtain naming norms in Italian (the reader is referred to Supplementary Material B for the norming of the materials).…”

“…We also included 40 five-object filler arrays, 32 arrays from Cimminella et al (2020), and 8 new arrays created using objects from the BOSS database (Brodeur et al, 2010(Brodeur et al, , 2014. The filler items ensured a balanced distribution of target detection response (i.e., 50% object present, 50% object absent); and were not systematically manipulated in semantic relatedness or visual saliency nor will they be analysed.…”