Linking signal detection theory and encoding models to reveal independent neural representations from neuroimaging data

Abstract

Many research questions in visual perception involve determining whether stimulus properties are represented and processed independently. In visual neuroscience, there is great interest in determining whether important object dimensions are represented independently in the brain. Unfortunately, most previous research has only vaguely defined what is meant by "independence," which hinders its precise quantification and testing. Here we present a new quantitative framework that links general recognition theory (GRT) and encoding models from computational neuroscience, focusing on a special form of independence: perceptual separability. Without loss of generality, consider the special case in which stimuli vary along two stimulus dimensions, represented by A and B, each with only two levels indexed by i = 1, 2 for dimension A and j = 1, 2 for dimension B. A stimulus is represented by a combination of these dimension levels, A i B j .In the computational neuroscience literature, an encoding model is a formal representation of the relation between stimuli and the response of a number of channels (single neurons or neural populations), represented by r. The channel responses are assumed to be random variables, and thus the response of the model is characterized by a probability distribution p (r|A i B j , θ), where θ represents a set of parameters describing neural noise. Encoding separability of dimension A from dimension B holds when encoding of the value of A does not change with the stimulus' value on B. That is, if and only if, for all values of r and i:( 1) The term neural decoding refers both to a series of methods used by researchers to extract information about a stimulus from neural data and to the mechanisms used by readout neurons to extract similar information. If a dimension is encoded by N channels, then the decoded estimate of a dimensional value is = g (r), where g () is a function from R N to R.Because r is a random vector, the decoded value is a random value that follows a probability distribution p  |A i B j , θ . Decoding separability of dimension A from dimension B holds when the distribution of decoded values of A does not change with the value of B in the stimulus-that is, if and only if, for all values of and i:It can be shown that encoding separability and decoding separability are related as summarized in Figure 1. In addition, when decoding separability is measured through the L1 distance between kernel density estimates of p  |A i B j , θ , the relations in Figure 1 hold even if decoding is performed on indirect measures of neural activity contaminated with error, as those obtained through fMRI. Figure 1 entails that when decoding separability is measured and fails, one can make the valid inference that encoding separability fails as well, but when decoding separability holds, only weak evidence of encoding separability holding has been obtained. Encoding Separability Decoding SeparabilityHolds Fails Holds Fails Figure 1 Importantly, it is possible to link these ideas to GRT by assumi...