Improving the validity of neuroimaging decoding tests of invariant and configural neural representation

Abstract

Many research questions in sensory neuroscience involve determining whether the neural representation of a stimulus property is invariant to some irrelevant stimulus change (e.g., viewpoint-invariant face representation, or modality-invariant object representations). Most neuroimaging studies have studied invariance using operational tests that have only face validity, of which the most popular in recent years is the cross-classification test. A recently proposed theoretical framework suggests that operational tests of invariance commonly used in the neuroimaging literature, such as cross-classification, might lead to invalid conclusions. Here, we provide empirical and computational evidence supporting this theoretical insight. In our empirical study, we use encoding of orientation and spatial position in primary visual cortex as a case study, as previous research has established that these properties are not encoded in an invariant way. In a functional MRI study with human participants of both sexes, we show that the cross-classification test produces false positives, in many cases leading to the conclusion that orientation is encoded invariantly from spatial position, and that spatial position is encoded invariantly from orientation, in primary visual cortex. The results of two simulations further suggest that the test can lead to the conclusion of invariance when no sensible definition of invariance holds at the neural level, and that encoding strategies known to be used in cortex may easily lead to such false positives. On the other hand, we show that it is possible to provide evidence against invariance (i.e., contextdependent or configural encoding) through appropriate theory-driven decoding tests.