The Influence of Behavioral Relevance on the Processing of Global Scene Properties: An ERP Study

Hansen, Natalie; Noesen, Birken; Nador, Jeff; Harel, Assaf

doi:10.1167/19.8.57

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While this generally held true for the nine models used here, it should be noted that the gist features (Oliva & Torralba, 2001) were an exception (see Figure 1). Therefore, we have refrained from strongly interpreting results for that model, particularly the observation that this feature was not significantly predictive of the behavioral RDM (see Table 1) given previous reports that gist features can strongly influence categorization behavior (M. ), vERPs (Hansen, Noesen, Nador, & Harel, 2018), MEG patterns (Ramkumar et al, 2016), and fMRI activation patterns (Watson et al, 2017(Watson et al, , 2014.…”

Section: Discussionmentioning

confidence: 99%

Disentangling the Independent Contributions of Visual and Conceptual Features to the Spatiotemporal Dynamics of Scene Categorization

Greene

Hansen

2020

Preprint

View full text Add to dashboard Cite

Human scene categorization is characterized by its remarkable speed. While many visual and conceptual features have been linked to this ability, significant correlations exist between feature spaces, impeding our ability to determine their relative contributions to scene categorization. Here, we employed a whitening transformation to decorrelate a variety of visual and conceptual features and assess the time course of their unique contributions to scene categorization. Participants (both sexes) viewed 2,250 full-color scene images drawn from 30 different scene categories while having their brain activity measured through 256-channel EEG. We examined the variance explained at each electrode and time point of visual event-related potential (vERP) data from nine different whitened encoding models. These ranged from low-level features obtained from filter outputs to high-level conceptual features requiring human annotation. The amount of category information in the vERPs was assessed through multivariate decoding methods. Behavioral similarity measures were obtained in separate crowdsourced experiments. We found that all nine models together contributed 78% of the variance of human scene similarity assessments and was within the noise ceiling of the vERP data. Low-level models explained earlier vERP variability (88 ms post-image onset), while high-level models explained later variance (169 ms). Critically, only high-level models shared vERP variability with behavior. Taken together, these results suggest that scene categorization is primarily a high-level process, but reliant on previously extracted low-level features. Significance StatementIn a single fixation, we glean enough information to describe a general scene category. Many types of features are associated with scene categories, ranging from low-level properties such as colors and contours, to high-level properties such as objects and attributes. Because these properties are correlated, it is difficult to understand each property's unique contributions to scene categorization. This work uses a whitening transformation to remove the correlations between features and examines the extent to which each feature contributes to visual eventrelated potentials (vERPs) over time. We found that low-level visual features contributed first, but were not correlated with categorization behavior. High-level features followed 80 ms later, providing key insights into how the brain makes sense of a complex visual world.

show abstract