Decoding sounds depicting hand-object interactions in primary somatosensory cortex

Abstract

Neurons, even in earliest sensory regions of cortex, are subject to a great deal of contextual influences from both within and across modality connections. Recently we have shown that cross-modal connections from vision to primary somatosensory cortex (SI) transmit contentspecific information about familiar visual object categories. In the present work, we investigated whether SI would also contain content-specific information about sounds depicting familiar hand-object interactions (e.g. bouncing a ball). In a rapid event-related functional magnetic resonance imaging (fMRI) experiment, participants listened attentively to multiple exemplars from three sound categories: hand-object interactions, and control categories of pure tones and animal vocalizations, while performing a one-back repetition counting task. Multi-voxel pattern analysis revealed significant decoding of different handobject sounds within bilateral post-central gyrus (PCG), whilst no significant decoding was found for either control category. Crucially, in the hand-sensitive voxels defined from an independent tactile localizer, decoding accuracies were significantly higher for decoding hand-object sounds compared to both control categories in left PCG. Our findings indicate that hearing sounds depicting familiar hand-object interactions elicit different patterns of activity in SI, despite the complete absence of tactile stimulation. Thus cross-modal connections from audition to SI transmit content-specific information about sounds depicting familiar hand-object interactions.