The timecourse of inter-object contextual facilitation

Abstract

High-level vision is frequently studied at the level of either individual objects or full scenes. An intermediate level of visual organisation that has received less attention is the ″object constellation″, defined here as a familiar configuration of contextually-associated objects (e.g., plate + spoon). Recent work has shown that information from multiple objects can be integrated to support observers′ high-level understanding of a ″scene″. Here we used EEG to test when the visual system integrates information across objects to support representations of real-world size. To this end, we briefly presented masked object constellations consisting of either large (e.g., chair + table) or small (e.g., plate + spoon) object silhouettes, while independently varying retinal size. As a control, observers also saw each object silhouette presented in isolation. Behavioural results showed that observers recognized the objects′ real-world size more easily when the silhouettes appeared in pairs than when they appeared in isolation. Representational similarity analysis of EEG data revealed that neural activity patterns captured information about the real-world size of object constellations from ~200 ms after stimulus onset. This representation was stronger for, and specific to, object pairs as compared to single objects, and remained significant after regressing out visual similarity models derived from computational models. These results reveal the neural time course of real-world size extracted from familiar constellations of objects. More generally, they provide evidence for inter-object facilitation of visual processing, leading to a qualitatively different high-level representation of object pairs than single objects.

Object representations reflect hierarchical scene structure and depend on high-level visual, semantic, and action information.

“…Its architecture approximates the hierarchical structure of the ventral visual system (layer blocks: V1, V2, V4, IT). We incorporated CORnet-S for two reasons: first, we wanted to account for similarities in low-level visual features between objects of the same phrase or scene 11,20,20,26 . Second, we wanted to know whether a state-of-the-art deep neural network (DNN) trained on object classification would represent scene grammar like structure in complex, high-dimensional visual feature spaces.…”

“…Here, we modelled crossclassification confusion matrices from representational dissimilarity matrices (RDMs) obtained from a range of encoding models, each model representing hypotheses about the shared feature space. We included models that quantified similarities in low-level visual features 11,20,20,26 and more abstract, high-level models quantifying semantic and action related similarity between our conditions as well as real-world co-occurrence statistics. We expected high-level features to explain a significant amount of variance in classifier confusions while accounting for similarities in low-level features.…”