Spatiotemporal Dynamics of Sound Representations reveal a Hierarchical Progression of Category Selectivity

Abstract

As the human brain transforms incoming sounds, it remains unclear whether semantic meaning is assigned via distributed, domain-general architectures or specialized hierarchical streams. Here we show that the spatiotemporal progression from acoustic to semantically dominated representations is consistent with a hierarchical processing scheme. Combining magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) patterns, we found superior temporal responses beginning ~80 ms post-stimulus onset, spreading to extratemporal cortices by ~130 ms.Early acoustically-dominated representations trended systematically toward semantic category dominance over time (after ~200 ms) and space (beyond primary cortex). Semantic category representation was spatially specific: vocalizations were preferentially distinguished in temporal and frontal voice-selective regions and the fusiform face area; scene and object sounds were distinguished in parahippocampal and medial place areas. Our results are consistent with an extended auditory processing hierarchy in which acoustic representations give rise to multiple streams specialized by category, including areas typically considered visual cortex.

The Spatiotemporal Neural Dynamics of Intersensory Attention Capture of Salient Stimuli: A Large-Scale Auditory-Visual Modeling Study

“…For example, among the stimuli used by Leaver and Rauschecker (2010) were human speech and musical instrument sounds. Lowe et al (2020) employed both MEG and fMRI to study categorization of human, animal and object sounds. Depending on the exact experimental design, sound source information may require long-term memory.…”

Neural representation of spatial and non-spatial auditory attention in EEG signals

“…The RSA approach to study neural representation (Kriegeskorte et al, 2008) has been applied in multiple domains in neuroscience research, including visual object recognition (Connolly et al, 2012;Cichy et al, 2014Cichy et al, , 2016, visual object processing (Kaneshiro et al, 2015), scene perception (Cichy and Teng, 2017), audiovisual integration (Cecere et al, 2017), and semantic categorization of sound (Lowe et al, 2020). In these studies, researchers presented large sets of different visual or auditory stimuli to participants and studied neural representation of the object categories (e.g., human faces, animate objects, etc.)…”