Principles of Temporal Processing Across the Cortical Hierarchy

Abstract

The world is richly structured on multiple spatiotemporal scales. In order to represent spatial structure, many machine-learning models repeat a set of basic operations at each layer of a hierarchical architecture. These iterated spatial operations - including pooling, normalization and pattern completion - enable these systems to recognize and predict spatial structure, while robust to changes in the spatial scale, contrast and noisiness of the input signal. Because our brains also process temporal information that is rich and occurs across multiple time scales, might the brain employ an analogous set of operations for temporal information processing? Here we define a candidate set of temporal operations, and we review evidence that they are implemented in the mammalian cerebral cortex in a hierarchical manner. We conclude that multiple consecutive stages of cortical processing can be understood to perform temporal pooling, temporal normalization and temporal pattern completion.

Constructing and Forgetting Temporal Context in the Human Cerebral Cortex

“…In HAT, if prior context can be successfully compressed with new input, then the context is preserved, but if prior context and new input are incompatible (prediction error), then the context is overwritten. This surprise-driven gating mechanism is consistent with evidence for pattern violations being signaled independently at multiple levels of cortical processing (Bekinschtein et al, 2009;Himberger et al, 2018;Wacongne et al, 2011).…”

No evidence for functional distinctions across fronto-temporal language regions in their temporal receptive windows

“…These findings situate the core language network within the context of a cortical hierarchy of integration timescales (Himberger et al, 2018). The common functional profile shared by temporal and inferior frontal language regions occupies a particular stage within this broader hierarchy, which is located, as expected, downstream from auditory regions and upstream from the episodic network.…”

Intrinsic dynamics enhance temporal stability of stimulus representation along rodent visual cortical hierarchies

“…A second key question addressed by our study is the role of intrinsic processing in establishing the persistency of neuronal representations along an object-processing pathway. In both primates and rodents, intrinsic temporal scales have been found to increase along various cortical hierarchies (Chaudhuri et al 2015;Himberger et al 2018;Murray et al 2014;Runyan et al 2017). However, it is unknown whether a similar increase takes place along the ventral stream and, if so, how such increase may combine with the growing stability of stimulus-driven responses we have discussed in the previous section.…”

“…Recently it has been shown that models of the ventral stream based on deep convolutional neural networks can be improved in their predictive power for perception and neural activity by including adaptation mechanisms (Vinken et al 2019) and recurrent processing (Kar et al 2019;Kietzmann et al 2019;Tang et al 2018). Moreover, a progressive increase of the importance of intrinsic processing along the ventral stream may be expected, given that intrinsic temporal scales increase along various cortical hierarchies in primates and rodents (Chaudhuri et al 2015;Himberger et al 2018;Murray et al 2014;Runyan et al 2017). However, it is not known how the interaction between invariant encoding of object information and intrinsic processing unfolds along the ventral stream, and whether or not the net result is an ordered progression of temporal scales of neural processing.…”