A Common, High-Dimensional Model of the Representational Space in Human Ventral Temporal Cortex

Abstract

Summary We present a high-dimensional model of the representational space in human ventral temporal (VT) cortex in which dimensions are response-tuning functions that are common across individuals and patterns of response are modeled as weighted sums of basis patterns associated with these response-tunings. We map response pattern vectors, measured with fMRI, from individual subjects’ voxel spaces into this common model space using a new method, ‘hyperalignment’. Hyperalignment parameters based on responses during one experiment – movie-viewing – identified 35 common response-tuning functions that captured fine-grained distinctions among a wide range of stimuli in the movie and in two category perception experiments. Between-subject classification (BSC, multivariate pattern classification based on other subjects’ data) of response pattern vectors in common model space greatly exceeded BSC of anatomically-aligned responses and matched within-subject classification. Results indicate that population codes for complex visual stimuli in VT cortex are based on response-tuning functions that are common across individuals.

Object-Specific Semantic Coding in Human Perirhinal Cortex

“…As different categories of object tend to have different statistical properties associated with their semantic features (e.g., animals have many frequently co-occurring features that are shared by other animals), categorical effects can emerge. One promising approach to untangling potential representational principles in posterior VTC is to represent activation patterns in a high-dimensional space, that may be used to test whether categorical effects are emergent from a combination of noncategory factors (Op de Beeck et al, 2008;Haxby et al, 2011).…”

Data-driven approaches in the investigation of social perception

“…Moreover, anatomy-based alignment does not capture the idiosyncratic individual variability of coarse topographic features, such as the location, size and conformation of the borders of functional areas such as retinotopically organized early visual areas, motion-sensitive MT or category-selective areas in ventral temporal cortex. A common model of the functional architecture that captures these features-fine-scale patterns of activity and individual variability of coarse-scale features-has been developed using a new algorithm, hyperalignment, and achieves broad general validity by estimating model parameters based on responses to a complex, dynamic, naturalistic stimulus, such as a fulllength movie [47,51,52]. The elements of this model are a common, high-dimensional representational space and individual transformation matrices that project data from idiosyncratic, individual anatomic spaces into the common model space.…”

Emerging methods for conceptual modelling in neuroimaging

“…This can have the additional benefit of increasing the generality of the results obtained -a case in point is the datadriven decomposition of fMRI brain activity in Mitchell et al (2008), which used a webcorpus derived semantic space to identify neural signatures for individual components of meaning, and so make the models capable of accounting for concepts/stimuli that have not been encountered during training (so called "zero-shot" learning). While multivariate approaches have a potential drawback in their ability to generalise across sessions, participants and tasks 2 , proposals have also been made by ourselves and others (Kriegeskorte et al 2008a;Haxby et al 2011;Akama et al 2014) to deal with this effectively. Over the past few years a significant progress is being made in this field, and it is worthwhile to emphasize that the involvement of widespread cortical areas were elucidated by using Multivariate approaches to multilingual processing Pliatsikas et al, 2016).…”

“…One solution to this issue to that has already been introduced is the hyperalignment method from Haxby et al (2011). This uses a parallel time-courses of fMRI data from two or more participants/sessions, driven by an engaging task such as movie watching.…”

“…This can make generalisations over multiple participants and session more challenging (conventional analyses typically correct for this with strategies involving temporal and spatial averaging and smoothing). Haxby et al (2011) introduced a new method for cross-participant normalisation, based on richer, more ecologically valid stimulus: popular film. The hyperalignment method also used regression-based techniques to identify a set of voxels in one participant whose collective activity corresponded to that of the participant group.…”