Humans effortlessly make quick and accurate perceptual decisions about the nature of their immediate visual environment, such as the category of the scene they face. Previous research has revealed a rich set of cortical representations potentially underlying this feat. However, it remains unknown which of these representations are suitably formatted for decision-making. Here, we approached this question empirically and computationally, using neuroimaging and computational modelling. For the empirical part, we collected electroencephalography (EEG) data and reaction times from human participants during a scene categorization task (natural vs. man-made). We then related neural representations to behaviour using a multivariate extension of signal detection theory. We observed a correlation specifically between ~100 ms and ~200 ms after stimulus onset, suggesting that the neural scene representations in this time period are suitably formatted for decision-making. For the computational part, we evaluated a recurrent convolutional neural network (RCNN) as a model of brain and behaviour. Unifying our previous observations in an image-computable model, the RCNN predicted well the neural representations, the behavioural scene categorization data, as well as the relationship between them. Our results identify and computationally characterize the neural and behavioural correlates of scene categorization in humans.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.