Beyond linear regression: mapping models in cognitive neuroscience should align with research goals

Abstract

Many cognitive neuroscience studies use large feature sets to predict and interpret brain activity patterns. Feature sets take many forms, from human stimulus annotations to representations in deep neural networks. Of crucial importance in all these studies is the mapping model, which defines the space of possible relationships between features and neural data. Until recently, most encoding and decoding studies have used linear mapping models. Increasing availability of large datasets and computing resources has recently allowed some researchers to employ more flexible nonlinear mapping models; however, the question of whether nonlinear mapping models can yield meaningful scientific insights remains debated. Here, we discuss the choice of a mapping model in the context of three overarching desiderata: predictive accuracy, interpretability, and biological plausibility. We show that these desiderata do not map cleanly onto the linear/nonlinear divide; instead, each desideratum can refer to multiple research goals, each of which imposes its own constraints on the mapping model. Moreover, we argue that, instead of categorically treating the mapping models as linear or nonlinear, researchers should report the complexity of these models. We show that, in many cases, complexity provides a more accurate reflection of restrictions imposed by various research goals and outline several complexity metrics that can be used to effectively evaluate mapping models.

Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions

“…This assumption is common in systems neuroscience, but could obscure aspects of a model representation that deviate markedly from those of the brain, because the linear mapping picks out only the model features that are predictive of brain responses. There is ample evidence that deep neural network models tend to rely partially on different features than humans 73, 74 , and have partially distinct invariances 35, 75 for reasons that remain unclear. Encoding model analyses likely mask the presence of such discrepant model properties.…”

“…It will also be important to use additional means of model evaluation, such as model-matched stimuli 35, 61, 75 , stimuli optimized for the model’s predicted response 76, 88, 89 , or directly substituting brain responses into models 90 . And ultimately, analyses such as these need to be related to more fine-grained anatomy and brain response measurements.…”

“…The conclusions here are also limited by the two metrics of model-brain similarity that we used. The regression-based metric of explained variance is based on the assumption that representational similarity can be meaningfully assessed using a linear mapping between responses to natural stimuli 46, 108, 109 . This assumption is common in systems neuroscience, but could obscure aspects of a model representation that deviate markedly from those of the brain, because the linear mapping picks out only the model features that are predictive of brain responses.…”

Speech prosody enhances the neural processing of syntax

“…A driving force underlying this paradigm change is the increasingly flexible and powerful computational tools that are now becoming available to model human speech. Indeed, the use of machine learning techniques to statistically assess (and control for) different linguistic features has led to a new line of research that has the aim of modeling complex cognitive and neural functions in ecologically valid settings 49 . Importantly, such model-based encoding and decoding approaches allow results to be interpretable in light of proposed underlying cognitive and/or neural mechanisms 50 .…”

“…In the present study, we made use of naturalistic stimuli together with brain decoding 49 to fill a knowledge gap in how the cortical representation of syntax can be modulated by prosody to facilitate speech processing . We used machine learning techniques to compute, on the one hand, the syntactic phrase boundaries, and, on the other hand, the prosodic boundaries of a speech corpus (TED talks).…”