Brain Reading Using Full Brain Support Vector Machines for Object Recognition: There Is No “Face” Identification Area

Abstract

Over the past decade, object recognition work has confounded voxel response detection with potential voxel class identification. Consequently, the claim that there are areas of the brain that are necessary and sufficient for object identification cannot be resolved with existing associative methods (e.g., the general linear model) that are dominant in brain imaging methods. In order to explore this controversy we trained full brain (40,000 voxels) single TR (repetition time) classifiers on data from 10 subjects in two different recognition tasks on the most controversial classes of stimuli (house and face) and show 97.4% median out-of-sample (unseen TRs) generalization. This performance allowed us to reliably and uniquely assay the classifier's voxel diagnosticity in all individual subjects' brains. In this two-class case, there may be specific areas diagnostic for house stimuli (e.g., LO) or for face stimuli (e.g., STS); however, in contrast to the detection results common in this literature, neither the fusiform face area nor parahippocampal place area is shown to be uniquely diagnostic for faces or places, respectively.

Discriminating Taxonomic Categories and Domains in Mental Simulations of Concepts of Varying Concreteness

“…Data about the organization of conceptual knowledge in the brain coming from patients with semantic deficits Patterson, Nestor, & Rogers, 2007;Damasio, Tranel, Grabowski, Adolphs, & Damasio, 2004;Vinson, Vigliocco, Cappa, & Siri, 2003;Caramazza & Shelton, 1998;Warrington & Shallice, 1984) or collected from healthy patients using fMRI (Malach, Levy, & Hasson, 2002;Martin & Chao, 2001;Ishai, Ungerleider, Martin, Schouten, & Haxby, 1999) have proven an essential source of evidence for our understanding of conceptual representations, particularly when analyzed using machine learning methods (e.g., Connolly et al, 2012;Chang, Mitchell, & Just, 2010;Just, Cherkassky, Aryal, & Mitchell, 2010;Hanson & Halchenko, 2008;Kriegeskorte, Mur, & Bandettini, 2008;Kriegeskorte, Mur, Ruff, et al, 2008;Mitchell et al, 2008;Shinkareva, Mason, Malave, Wang, & Mitchell, 2008;Kamitani & Tong, 2005;OʼToole, Jiang, Abdi, & Haxby, 2005;Hanson, Matsuka, & Haxby, 2004;Haxby et al, 2001). Most of this work, however, has focused on a narrow range of conceptual categories, primarily concrete concepts such as animals, plants, tools, etc.…”

Commonality of neural representations of words and pictures

“…This approach has the potential to be particularly useful in determining how semantic information about objects is represented in the cerebral cortex. Using multi-voxel pattern analysis, previous studies succeeded in identifying the cognitive states associated with viewing categories of visually depicted objects (Carlson et al, 2003;Cox and Savoy, 2003;Hanson and Halchenko, 2007;Hanson et al, 2004;Haxby et al, 2001;O'Toole et al, 2005;Polyn et al, 2005;Shinkareva et al, 2008), objects presented in the combined word (noun) and picture form , or objects referred to by a written word (Just et al, 2010). In this work we explore whether the patterns of brain activity associated with thinking about concrete objects are dependent on stimulus presentation format, whether an object is referred to by a written or pictorial form.…”

Exploring commonalities across participants in the neural representation of objects

“…This approach has the potential to be particularly useful in determining how semantic information about objects is represented in the cerebral cortex. Using multivoxel pattern analysis, previous studies succeeded in identifying the cognitive states associated with viewing categories of objects [Carlson et al, 2003;Cox and Savoy, 2003;Hanson and Halchenko, 2007;Hanson et al, 2004;Haxby et al, 2001;O'Toole et al, 2005;Polyn et al, 2005]. Moreover, the category of an object that a participant was viewing [Shinkareva et al, 2008[Shinkareva et al, , 2011 or a concrete noun that a participant was reading [Just et al, 2010;Shinkareva et al, 2011] can be identified based only on other participants' characteristic neural activation patterns, establishing the commonality in how different people's brains represent the same object.…”