Real-life activities, such as watching a movie or engaging in conversation, unfold over many minutes. In the course of such activities, the brain has to integrate information over multiple time scales. We recently proposed that the brain uses similar strategies for integrating information across space and over time. Drawing a parallel with spatial receptive fields, we defined the temporal receptive window (TRW) of a cortical microcircuit as the length of time before a response during which sensory information may affect that response. Our previous findings in the visual system are consistent with the hypothesis that TRWs become larger when moving from low-level sensory to high-level perceptual and cognitive areas. In this study, we mapped TRWs in auditory and language areas by measuring fMRI activity in subjects listening to a real-life story scrambled at the time scales of words, sentences, and paragraphs. Our results revealed a hierarchical topography of TRWs. In early auditory cortices (A1ϩ), brain responses were driven mainly by the momentary incoming input and were similarly reliable across all scrambling conditions. In areas with an intermediate TRW, coherent information at the sentence time scale or longer was necessary to evoke reliable responses. At the apex of the TRW hierarchy, we found parietal and frontal areas that responded reliably only when intact paragraphs were heard in a meaningful sequence. These results suggest that the time scale of processing is a functional property that may provide a general organizing principle for the human cerebral cortex.

How are objects represented in the human visual cortex? Two conflicting theories suggest either a holistic representation, in which objects are represented by a collection of object templates, or a part-based representation, in which objects are represented as collections of features or object parts. We studied this question using a gradual object-scrambling paradigm in which pictures of objects (faces and cars) were broken in a stepwise manner into an increasing number of blocks. Our results reveal a hierarchical axis oriented anterior--posteriorly in the organization of ventral object-areas. Along this axis, representations are arranged in bands of increasing sensitivity to image scrambling. The axis starts in early visual areas through retinotopic areas V4/V8 and continues into the lateral-occipital sulcus dorsally and the posterior fusiform girus ventrally, corresponding together to the previously described object-related lateral occipital complex (LOC). Regions showing the highest sensitivity to scrambling tended to be located at the most anterior-lateral regions of the complex. In these more anterior regions, breaking the images into 16 parts produced a significant reduction in activation. Interestingly, activation was not affected when images were cut in two halves, either horizontally or vertically. Car images generally produced a weaker activation compared to faces in the lateral occipital complex but showed the same tendency of increased scrambling sensitivity along the anterior--posterior axis. These results suggest the existence of a hierarchical axis along ventral occipito-temporal object-areas, in which the neuronal properties shift from sensitivity to local object features to a more global and holistic representation.

How similar are the brains of listeners who hear the same content expressed in different languages? We directly compared the functional magnetic resonance (fMRI) brain responses of English speakers and Russian speakers who listened to a real-life Russian narrative and its English translation. During the translation we tried to preserve the content of the narrative while reducing the structural similarities across languages. The story evoked similar brain responses across languages, which were invariant to the structural changes, beginning just outside early auditory areas and extending through temporal, parietal and frontal cerebral cortices. Surprisingly, the inter-language similarity in these areas is nearly as strong as the similarity of the brain responses within each language group. The present results demonstrate that the human brain processes real-life information in a manner that is largely insensitive to the language in which that information is conveyed. The methods introduced here can potentially be used to quantify the transmission of meaning across cultural and linguistic boundaries.

The ability of the human visual system to recognize partially occluded objects is a striking feat, which has received extensive psychophysical documentation. Here we studied the manifestation of completion effects in the functional magnetic resonance imaging (fMRI) activation of high-order object areas (the lateral occipital complex - LOC). Subjects were presented with three types of images: (i) whole line drawings of animal or unfamiliar shapes ('whole'); (ii) the same shapes, occluded by parallel stripes which occupied roughly half of the surface area of the images ('grid'); and (iii) the same stripes, 'scrambled' so that the relative position of the regions between the stripes was changed while the local feature structure remained intact. Behavioral measurements showed a high degree of object completion in the 'grid' condition, but not in the 'scrambled' condition. The fMRI results show a significantly higher activation to the 'grid' images compared to the 'scrambled' images. This enhanced activation indicates the operation of non-local completion effects, since the local features in both sets of images were the same. The cortical regions showing the highest 'completion' effects co-localized with regions in the LOC which showed the highest activation to the 'whole' images compared to the 'scrambled' images. Activation in early retinotopic areas was similar in both the 'grid' and the 'scrambled' conditions. Our results point to the LOC as a central site in which object completion effects are manifested.

Different brain areas integrate information over different timescales, and this capacity to accumulate information increases from early sensory areas to higher order perceptual and cognitive areas. It is currently unknown whether the timescale capacity of each brain area is fixed or whether it adaptively rescales depending on the rate at which information arrives from the world. Here, using functional MRI, we measured brain responses to an auditory narrative presented at different rates. We asked whether neural responses to slowed (speeded) versions of the narrative could be compressed (stretched) to match neural responses to the original narrative. Temporal rescaling was observed in early auditory regions (which accumulate information over short timescales) as well as linguistic and extra-linguistic brain areas (which can accumulate information over long timescales). The temporal rescaling phenomenon started to break down for stimuli presented at double speed, and intelligibility was also impaired for these stimuli. These data suggest that 1) the rate of neural information processing can be rescaled according to the rate of incoming information, both in early sensory regions as well as in higher order cortexes, and 2) the rescaling of neural dynamics is confined to a range of rates that match the range of behavioral performance.

The role of early visual experience in the establishment of human high-order visual areas is poorly understood. Here we investigated this issue using human amblyopia--a developmental visual disorder, which manifests a central vision (acuity) deficit. Previous fMRI studies of amblyopes have described abnormal functional activations in early retinotopic areas. Here we report the surprising finding of a selective object-related abnormality in high-order occipitotemporal cortex. Specifically, we found that face-related cortical areas show a severe disconnection from the amblyopic eye, while building-related regions remain essentially normal. The selectivity of the deficit highlights the differential computations performed in the different object-related areas and is compatible with the suggested association of face regions with analysis of fine detail.

Music, like speech, is a complex auditory signal that contains structures at multiple timescales, and as such is a potentially powerful entry point into the question of how the brain integrates complex streams of information. Using an experimental design modeled after previous studies that used scrambled versions of a spoken story (Lerner et al., 2011) and a silent movie (Hasson et al., 2008), we investigate whether listeners perceive hierarchical structure in music beyond short (~6 s) time windows and whether there is cortical overlap between music and language processing at multiple timescales. Experienced pianists were presented with an extended musical excerpt scrambled at multiple timescales—by measure, phrase, and section—while measuring brain activity with functional magnetic resonance imaging (fMRI). The reliability of evoked activity, as quantified by inter-subject correlation of the fMRI responses, was measured. We found that response reliability depended systematically on musical structure coherence, revealing a topographically organized hierarchy of processing timescales. Early auditory areas (at the bottom of the hierarchy) responded reliably in all conditions. For brain areas at the top of the hierarchy, the original (unscrambled) excerpt evoked more reliable responses than any of the scrambled excerpts, indicating that these brain areas process long-timescale musical structures, on the order of minutes. The topography of processing timescales was analogous with that reported previously for speech, but the timescale gradients for music and speech overlapped with one another only partially, suggesting that temporally analogous structures—words/measures, sentences/musical phrases, paragraph/sections—are processed separately.

No abstract