Understanding Human Object Vision: A Picture Is Worth a Thousand Representations

Abstract

Objects are the core meaningful elements in our visual environment. Classic theories of object vision focus upon object recognition and are elegant and simple. Some of their proposals still stand, yet the simplicity is gone. Recent evolutions in behavioral paradigms, neuroscientific methods, and computational modeling have allowed vision scientists to uncover the complexity of the multidimensional representational space that underlies object vision. We review these findings and propose that the key to understanding this complexity is to relate object vision to the full repertoire of behavioral goals that underlie human behavior, running far beyond object recognition. There might be no such thing as core object recognition, and if it exists, then its importance is more limited than traditionally thought. Expected final online publication date for the Annual Review of Psychology, Volume 74 is January 2023. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Deep problems with neural network models of human vision

“…These alternative models are very appealing but also more narrow in scope. Consider, for example, the simplicity with which the well-known ALCOVE model explains categorization (Kruschke, 1992), compared to the complex high-dimensional space that is the actual reality of the underlying representations (for a review, see Bracci & Op de Beeck, 2023). Note that we consider these alternatives to be an excellent way to obtain a conceptual understanding of a phenomenon, we all very much build on top of this pioneering work using conceptually elegant models with few parameters (e.g., .…”

“…The way forward is to build better models, including DNN-based models that take the complexity of human vision and cognition seriously (Bracci & Op de Beeck, 2023). As it has been since the very early days of AI, we need continuous interaction and exchange between disciplines and their expertise at all levels (cognitive and computational psychologists, computer vision scientists, philosophers of the mind, neuroscientists) to bring us towards a common goal of a humanlike AI that we understand mechanistically.…”

The representational hierarchy in human and artificial visual systems in the presence of object-scene regularities

“…In visual cortex, the domain model captures most of the variance in animal as well as in scene areas, but the dissimilarity matrices ( Fig 2A ) reveal a marked difference between the two sets of regions that likely reflect differential domain-specific object spaces [ 7 – 9 , 47 , 48 ]. This rich dimensionality can support the need of our brain to employ different representations for different behavioural needs [ 49 ]. For instance, in scene selective areas, the degree of navigational layout well characterizes its representational content which is relevant for naviation [ 7 , 8 ].…”

“…Together, this highlights the advantages of domain-specialized modules evolved by biological vision [ 51 ]. We suggest that future DCNNs that aim to capture the rich and diverse representational space found in VTC need to employ tasks that go beyond standard object recognition tasks and target the diverse computational goals our visual system supports [ 49 ].…”

Going after the bigger picture: Using high-capacity models to understand mind and brain

“…These alternative models are very appealing but also more narrow in scope. Consider, for example, the simplicity with which the well-known ALCOVE model explains categorization (Kruschke, 1992), compared to the complex high-dimensional space that is the actual reality of the underlying representations (for a review, see Bracci & Op de Beeck, 2023). Note that we consider these alternatives to be an excellent way to obtain a conceptual understanding of a phenomenon, we all very much build on top of this pioneering work using conceptually elegant models with few parameters (e.g., Ritchie & Op de Beeck, 2019).…”