Representational Similarity Analysis (RSA) has emerged as a popular method for relating representational spaces from human brain activity, behavioral data, and computational models. RSA is based on the comparison of rep- resentational dissimilarity matrices (RDM), which characterize the pairwise dissimilarities of all conditions across all features (e.g. fMRI voxels or units of a model). However, classical RSA treats each feature as equally important. This 'equal weights' assumption contrasts with the flexibility of multivariate decoding, which reweights individual features for predicting a target variable. As a consequence, classical RSA may lead researchers to underestimate the correspondence between a model and a brain region and, for model comparison, it may lead to selecting the inferior model. While previous work has suggested that reweighting can improve model selection in RSA, it has remained unclear to what extent these results generalize across datasets and data modalities. To fill this gap, the aim of this work is twofold: First, utilizing a range of publicly available datasets and three popular deep neural networks (DNNs), we seek to broadly test feature-reweighted RSA (FR-RSA) applied to computational models and reveal the extent to which reweighting model features improves RDM correspondence and affects model selection. Second, we propose voxel-reweighted RSA, a novel use case of FR-RSA that reweights fMRI voxels, mirroring the rationale of multivariate decoding of optimally combining voxel activity patterns. We find that reweighting individual model units (1) markedly improves the fit between model RDMs and target RDMs derived from several fMRI and behavioral datasets and (2) affects model selection, highlighting the importance of considering FR-RSA. For voxel-reweighted RSA, improvements in RDM correspondence were even more pronounced, demonstrating the utility of this novel approach. We additionally demonstrate that classical noise ceilings can be exceeded when FR-RSA is applied and propose an updated approach for their computation. Taken together, our results broadly validate the use of FR-RSA for improving the fit between computational models, brain and behavioral data, possibly allowing us to better adjudicate between competing computational models. Further, our results suggest that FR-RSA applied to brain measurement channels could become an important new method to assess the match between representational spaces.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.