Object responses are highly malleable, rather than invariant, with changes in object appearance

Abstract

theoretical frameworks of human vision argue that object responses remain stable, or 'invariant', despite changes in viewing conditions that can alter object appearance but not identity. Here, in a major departure from previous approaches that have relied on two-dimensional (2-D) images to study object processing, we demonstrate that changes in an object's appearance, but not its identity, can lead to striking shifts in behavioral responses to objects. We used inverse multidimensional scaling (MDS) to measure the extent to which arrangements of objects in a sorting task were similar or different when the stimuli were displayed as scaled 2-D images, three-dimensional (3-D) augmented reality (AR) projections, or real-world solids. We were especially interested in whether sorting behavior in each display format was based on conceptual (e.g., typical location) versus physical object characteristics. We found that 2-D images of objects were arranged according to conceptual (typical location), but not physical, properties. AR projections, conversely, were arranged primarily according to physical properties such as real-world size, elongation and weight, but not conceptual properties. Real-world solid objects, unlike both 2-D and 3-D images, were arranged using multidimensional criteria that incorporated both conceptual and physical object characteristics. our results suggest that object responses can be strikingly malleable, rather than invariant, with changes in the visual characteristics of the stimulus. The findings raise important questions about limits of invariance in object processing, and underscore the importance of studying responses to richer stimuli that more closely resemble those we encounter in real-world environments.Understanding how naturalistic stimuli are processed and represented in the human brain remains a major challenge for psychology, neuroscience and computer vision. Current theoretical frameworks of object vision posit that effective recognition requires invariance -that mental representations remain stable despite changes in visual cues that can alter an object's appearance but not its identity 1-5 . Support for representational invariance has been derived from behavioral 6-8 and neuroimaging studies 3,9 (although with some limitations, for example, depending on the retinal location of the stimulus 10,11 and task demands 12 ). Two-dimensional (2-D) images of objects have been shown to elicit size-invariant responses during recognition 13 , priming 7,14 , perceptual learning 15 , and visual search 16 . Similarly, neuroimaging studies have shown that responses in shape-selective regions within ventral occipito-temporal (vOT) cortex remain relatively constant despite changes in image size 17,18 , as well as changes in position 19 , viewpoint 4,9,11,15 and depth cues 18,20 . Reliable adaptation in ventral object areas across changes in the size of object images is observed in children by 5 to 10 years of age 21 . FMRI responses in vOT have also been found to remain constant across changes in t...

It's not all about looks: The role of object shape in parietal representations of manual tools

“…The effect of the object real-size in parietal cortex has been investigated in the context of object affordances (using real objects) and limited to small graspable objects (Cavina-Pratesi, Goodale, & Culham, 2007;Fabbri et al, 2017;Kourtis, Vandemaele, & Vingerhoets, 2018). However, recent evidence suggests that properties we extract from objects vary depending on the viewing format: physical properties (e.g., object size, weight, elongation) are relevant when watching real 3D objects, but not 2D images (Holler, Fabbri, & Snow, 2020). Consolidating these findings, we conclude that the manipulability effect observed in IPS is unlikely to be explained by imagined object physical size.…”

The Treachery of Images: Objects, Pictures, Words and the Role of Affordances in Similarity Judgements

“…This is the case even when considering experimental settings with different delivery apparatus. For instance, Holler & Snow (2020) found convergent results in an experimental setting where the real-world objects were handled physically whereas the pictures were manipulated on a computer screen. Our complimentary results were obtained in an experimental setting in which both real objects and pictures are physical manipulated.…”

“…1 Yet, there is very little research into the possible effects of affordances in this context and, crucially, in the similarity judgement paradigm for categorization tasks. For instance, while Holler and Snow (2020) recently found differences in how images and real-world objects are categorized, they did so without paying attention to the hypothesis that affordances may be responsible for those differences. Also, Castellini et al (2011) found that affordances potentiated object recognition but did not investigate the effect on similarity judgements.…”

Neural correlates of hand-object congruency effects during action planning

“…Previous studies have reported stronger affordance effects with real objects (Snow et al 2011, Gomez et al 2017) compared to 2D images of objects (Bub and Masson 2010, Squires et al 2016, Bub et al 2018). In a recent study, the categorisation of real objects led to the use of factors relating both conceptual and physical characteristics, whereas 2D-images were mostly categorised on the basis of conceptual characteristics alone (Holler et al 2020). Taken together, these differences might explain differences in activation patterns identified between the Grezes et al (2003) grasp categorisation study, and ours, which involved turning a real object with a reaching and wrist rotation movements.…”

“…Our results indicated functional interactions between 'dorso-dorsal' LPMd and ventral stream area LOCT in trials where hand posture and object orientation were incongruent, suggesting a direct integration between dorsal and ventral stream areas when preparing incongruent object manipulations (van Polanen and Davare 2015). Integrating ventral stream information during object manipulation dynamically with dorsal stream structures would suggest that affordance effects represent an influence of action semantics in tasks that do not require object categorisation or understanding, such as object manipulation (Creem and Proffitt 2001, Till et al 2014, van Elk et al 2014, Holler et al 2020.…”