Neural and behavioral signatures of the multidimensionality of manipulable object processing

Abstract

Understanding how we recognize objects requires unravelling the variables that govern the way we think about objects and the neural organization of object representations. A tenable hypothesis is that the organization of object knowledge follows key object-related dimensions. Here, we explored, behaviorally and neurally, the multidimensionality of object processing. We focused on within-domain object information as a proxy for the decisions we typically engage in our daily lives – e.g., identifying a hammer in the context of other tools. We extracted object-related dimensions from subjective human judgments on a set of manipulable objects. We show that the extracted dimensions are cognitively interpretable and relevant – i.e., participants are able to consistently label them, and these dimensions can guide object categorization; and are important for the neural organization of knowledge – i.e., they predict neural signals elicited by manipulable objects. This shows that multidimensionality is a hallmark of the organization of manipulable object knowledge.

Neural and behavioral similarity-driven tuning curves for manipulable objects

“…Areas of this cluster have been shown to be involved in aspects of high-level manipulable object processing. Specifically, they play a crucial role in processing object-specific visual information, such as information on shape, material, and surface properties (Grill-Spector and Malach, 2001;Hayworth and Biederman, 2006;Cant and Goodale, 2007;Cant et al, 2009;Pyles and Grossman, 2009;Gallivan et al, 2014;Freud et al, 2017b;Almeida et al, 2023a). Additionally, the medial fusiform gyrus has been associated with motor-relevant object information and functional grasping (Mahon et al, 2007;Valyear and Culham, 2010;Chen et al, 2016;Lee et al, 2019;Knights et al, 2022;Almeida et al, 2023a;Mahon and Almeida, 2024), whereas the parahippocampal and fusiform gyri have an important role in processing semantic knowledge and object function (Chao et al, 1999;Wheatley et al, 2005;Aminoff et al, 2013;Mandera et al, 2017;Kleineberg et al, 2018;Almeida et al, 2023a).…”

“…For Experiments 1 (a and b) and 2, we selected a set of manipulable objects previously used in our laboratory (Almeida et al, 2023a(Almeida et al, , 2023bValério et al, 2023). We chose to investigate manipulable objects because 1) these items represent everyday manmade objects that we constantly perceive and interact with; 2) these items encompass different sets of associated information, such as their function, the movements associated with their manipulation, specific structural features (e.g., a handle or rounded shape), and particular contexts in which we use them; 3) this category of objects can be independently impaired (or spared) in patients with brain lesions in the context of spared (or impaired) recognition of items from other domains of knowledge (e.g., animals or faces; for a review see Capitani et al, 2003); and 4) this category is well-suited for using a release from adaptation paradigm because the same object can be rendered in a wide diversity of materials, colors, and shapes (e.g., the various types of glasses).…”

“…Exploring how object knowledge is represented in the brain has led to the proposal of several hypotheses on the organization of information within the ventral and lateral temporal cortex (Kanwisher et al, 1997;Caramazza and Shelton, 1998;Chao et al, 1999;Haxby et al, 2001;Levy et al, 2001;Epstein, 2008;Kriegeskorte et al, 2008;Almeida et al, 2013;Konkle and Caramazza, 2013;Sha et al, 2015;Downing and Peelen, 2016;Mahon and Almeida, 2024 for a review see Grill-Spector and Weiner, 2014). For instance, it has been suggested that object knowledge is organized according to coarse domain distinctions, such as animate or inanimate (Caramazza and Shelton, 1998;Kriegeskorte et al, 2008), or more specific categories, such as faces, bodies, animals, manipulable objects, places, among others (Kanwisher et al, 1997;Chao et al, 1999;Epstein, 2008;Almeida et al, 2013;Downing and Peelen, 2016); or that the organization of object knowledge follows specific dimensions such as visual form (Haxby et al, 2001), eccentricity preferences (Levy et al, 2001;Arcaro et al, 2019), real-world object size (Konkle and Caramazza, 2013), animacy (Sha et al, 2015), or is organized according to multidimensional arrangements (Huth et al, 2012;Hebart et al, 2020;Fernandino et al, 2022;Almeida et al, 2023a). Importantly, several of these studies suggested that object similarity in the real world drives the organization of conceptual information in the brain (e.g., Kriegeskorte et al, 2008;Huth et al, 2012;Carlson et al, 2014;Bracci and Op de Beeck, 2016;<...…”

“…The number of studies evaluating the within-category organization of object knowledge is considerably lower -that is, studies investigating how information is represented among basic level members of a single domain (e.g., within the domain of manipulable objects, how is information represented among the different basic level members such as a knife and a pair of scissors; (Gotts et al, 2011;Connolly et al, 2012;Bruffaerts et al, 2013;Bracci et al, 2017;Almeida et al, 2023aAlmeida et al, , 2023bXu et al, 2023). This is at odds with the fact that the hardest and perhaps most common everyday decisions we tend to make (e.g., is the object in front of us a knife or a fork) typically rely on finergrained, within-category representations.…”

Contentopic mapping in ventral and dorsal association cortex: the topographical organization of manipulable object information

“…Specifically, we selected object-related dimensions that we obtained previously for a set of manipulable objects based on the subjective similarity between those manipulable objects on how they are manipulated in order to be used (see Methods) 1 . We chose the first two dimensions out of a 5-dimensional multidimensional scaling solution whose estimated distances between the tested objects were a good fit with the real distances (see Methods) 1 . That is, while these dimensions are still noisy and what they ultimately entail needs to be fully worked out, they are a relatively faithful description of the representational space we hold about how to manipulate objects.…”