Single-Unit Recordings in the Macaque Face Patch System Reveal Limitations of fMRI MVPA

Abstract

Multivariate pattern analysis (MVPA) of fMRI data has become an important technique for cognitive neuroscientists in recent years; however, the relationship between fMRI MVPA and the underlying neural population activity remains unexamined. Here, we performed MVPA of fMRI data and single-unit data in the same species, the macaque monkey. Facial recognition in the macaque is subserved by a well characterized system of cortical patches, which provided the test bed for our comparison. We showed that neural population information about face viewpoint was readily accessible with fMRI MVPA from all face patches, in agreement with single-unit data. Information about face identity, although it was very strongly represented in the populations of units of the anterior face patches, could not be retrieved from the same data. The discrepancy was especially striking in patch AL, where neurons encode both the identity and viewpoint of human faces. From an analysis of the characteristics of the neural representations for viewpoint and identity, we conclude that fMRI MVPA cannot decode information contained in the weakly clustered neuronal responses responsible for coding the identity of human faces in the macaque brain. Although further studies are needed to elucidate the relationship between information decodable from fMRI multivoxel patterns versus single-unit populations for other variables in other brain regions, our result has important implications for the interpretation of negative findings in fMRI multivoxel pattern analyses.

The Macaque Face Patch System: A Window into Object Representation

“…Moreover, the view decoding made mirror-symmetric mistakes in AL and AM, just as we had found earlier in the units (Fig. 2F) (Dubois et al 2015). This suggests that cells tuned to the same view are spatially clustered in each face patch.…”

Progress and challenges in probing the human brain

“…In addition, recent work combining electrophysiology and fMRI in non-human primates has demonstrated that the sensitivity of MVPA is limited by the spatial characteristics of the neuronal representations that code for particular features, such that some kinds of neuronal patterns may be more difficult to decode using MVPA than others 33 . Finally, it is important to stress that, like standard neuroimaging approaches, MVPA and RSA approaches do not inform about causal mechanisms.…”

“Hearing faces and seeing voices”: Amodal coding of person identity in the human brain

“…Why the anterior temporal poles were not observed in our results, despite their likely involvement in multimodal/amodal representations of individuals 1 , remains unclear. This is unlikely due to signal-to-noise ratio differences in anterior compared to posterior temporal regions 22 as we found above-chance classification for other classification schemes in some quite anterior temporal lobe clusters (Fig. 2).…”

“…Importantly, the limitations of a single stimulus per identity per modality discussed above do not bear on the interpretation of the cross-modal results: the training and test sets of fMRI data were acquired not just on different stimuli from the same identities, as has been performed by other 'unimodal' studies [19][20][21][22]24,25 but on different stimuli in different sensory modalities -arguably a more complex generalization problem than that posed by different person-related stimuli in the same modality 26 . So the cross-classification observed here is a 'true' identity classification, as opposed to the stimulus classification of our within-modality results.…”

“…Other studies suggest that amodal representations could rather emerge from cross-talk interactions between modality-specific areas 1 : voice and face-sensitive areas are not only connected via direct anatomical projections 15 but also functionally connected during familiar voice recognition 16 . Multi-voxel pattern analyses (MVPA) offer a powerful means of extracting information contained in distributed fMRI activity 17,18 : their enhanced sensitivity compared to classical univariate fMRI analyses has contributed to clarifying the neural correlates of unimodal face [19][20][21][22][23] or voice 24,25 identity processing. To our knowledge MVPA have not been used yet to examine the cerebral bases of multimodal person identification although they have provided exciting insights in the integration of emotional information across senses.…”