Compressive Temporal Summation in Human Visual Cortex

Abstract

Combining sensory inputs over space and time is fundamental to vision. Population receptive field models have been successful in characterizing spatial encoding throughout the human visual pathways. A parallel question-how visual areas in the human brain process information distributed over timehas received less attention. One challenge is that the most widely used neuroimaging method-fMRIhas coarse temporal resolution compared to the time-scale of neural dynamics. Here, via carefully controlled temporally modulated stimuli, we show that information about temporal processing can be readily derived from fMRI signal amplitudes in male and female subjects. We find that all visual areas exhibit sub-additive summation, whereby responses to longer stimuli are less than the linear prediction from briefer stimuli. We also find fMRI evidence that the neural response to two stimuli is reduced for brief interstimulus intervals (indicating adaptation). These effects are more pronounced in visual areas anterior to V1-V3. Finally, we develop a general model that shows how these effects can be captured with two simple operations: temporal summation followed by a compressive nonlinearity. This model operates for arbitrary temporal stimulation patterns and provides a simple and interpretable set of computations that can be used to characterize neural response properties across the visual hierarchy. Importantly, compressive temporal summation directly parallels earlier findings of compressive spatial summation in visual cortex describing responses to stimuli distributed across space. This indicates that for space and time, cortex uses a similar processing strategy to achieve higher-level and increasingly invariant representations of the visual world. Significance statementCombining sensory inputs over time is fundamental to seeing. Two important temporal phenomena are summation, the accumulation of sensory inputs over time, and adaptation, a response reduction for repeated or sustained stimuli. We investigated these phenomena in the human visual system using fMRI. We built predictive models that operate on arbitrary temporal patterns of stimulation using two simple computations: temporal summation followed by a compressive nonlinearity. Our new temporal compressive summation model captures (1) subadditive temporal summation, and (2) adaptation. We show that the model accounts for systematic differences in these phenomena across visual areas. Finally, we show that for space and time, the visual system uses a similar strategy to achieve increasingly invariant representations of the visual world.

A Network of Topographic Maps in Human Association Cortex Hierarchically Transforms Visual Timing-Selective Responses

“…Many models describe how monotonic or tuned responses could arise [6,38,39]. Early human visual cortex exhibits monotonic responses to visual event duration and frequency [1,2], from which tuned responses can be straightforwardly derived [40]. Our results show correlated and interacting selectivity for event duration and period (a measure of frequency).…”

“…Tested models included selectivity for temporal frequency; event duration and period; event period and occupancy (proportion of that period the event filled; i.e., period/duration); and event duration and inter-event interval (time from event offset to the next event onset; i.e., periodÀduration). We also compared logarithmic and linear response functions, and monotonic amplitude increases with occupancy (i.e., sustained neural response component) and/or event onset frequency (i.e., transient components) [1,2].…”

“…Here, AmplitudeRatio is the ratio of relative amplitudes of the sustained and transient components, and is the only free parameter. Next, the increase in response amplitude with frequency ( Figure S1E) or duration ( Figure S1F) may be sub-additive, so a compressive exponent is fit to capture the relationship between frequency and/or duration and response amplitude [2].…”

“…Humans can perceive sub-second event durations and frequencies, compare these across senses, and synchronize motor actions to them. Recent fMRI studies have demonstrated responses in human early visual areas that monotonically change with visual event timing, increasing with event frequency and duration [1,2]. For motor planning, macaque neurophysiological measurements demonstrate responses to motor event frequency in the macaque supplementary motor area, following both tuned and monotonic functions [3,4].…”

Reducing saccadic artifacts and confounds in brain imaging studies through experimental design

“…A task that focuses attention continuously at a center of fixation was a trivial candidate for this goal because such tasks were previously shown to reduce saccade rate (Pastukhov & Braun, ) and are often used in visual studies for other purposes. For instance, some fMRI studies use foveal tasks to focus attention at the center while measuring brain responses to task‐irrelevant peripheral stimulation (e.g., Goense & Logothetis, ; Watanabe et al, ; Yuval‐Greenberg & Heeger, ; Zhou, Benson, Kay, & Winawer, ). These studies demonstrated the plausibility of studying brain responses to task‐irrelevant parafoveal stimulation while attention is engaged in the fovea.…”