Coordinated multiplexing of information about separate objects in visual cortex

Jun, Na Young; Ruff, Douglas A.; Kramer, Lily E.; Bowes, Brittany; Tokdar, Surya T.; Cohen, Marlene R.

doi:10.1101/777912

Cited by 3 publications

(14 citation statements)

References 86 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our work focuses on one aspect of vision-the encoding of visual features; an exciting future direction is to combine encoding models with models of trial-to-trial variability [86,87,88,89,90], adaptation [91], spatial attention [92,93,94,95], arousal [96,97], visual crowding [98,99], saccade planning [100,101,102], perceptual learning [103], among other functions of V4 [27]. Overall, our compact models demonstrate that the task-driven computational models of the visual cortex are needlessly large.…”

Section: Discussionmentioning

confidence: 95%

Compact deep neural network models of visual cortex

Cowley,

Stan,

Pillow

et al. 2023

Preprint

View full text Add to dashboard Cite

A powerful approach to understanding the computations carried out in visual cortex is to develop models that predict neural responses to arbitrary images. Deep neural network (DNN) models have worked remarkably well at predicting neural responses [1, 2, 3], yet their underlying computations remain buried in millions of parameters. Have we simply replaced one complicated systemin vivowith anotherin silico? Here, we train a data-driven deep ensemble model that predicts macaque V4 responses ∼50% more accurately than currently-used task-driven DNN models. We then compress this deep ensemble to identifycompactmodels that have 5,000x fewer parameters yet equivalent accuracy as the deep ensemble. We verified that the stimulus preferences of the compact models matched those of the real V4 neurons by measuring V4 responses to both ‘maximizing’ and adversarial images generated using compact models. We then analyzed the inner workings of the compact models and discovered a common circuit motif: Compact models share a similar set of filters in early stages of processing but then specialize by heavily consolidating this shared representation with a precise readout. This suggests that a V4 neuron’s stimulus preference is determined entirely by its consolidation step. To demonstrate this, we investigated the compression step of a dot-detecting compact model and found a set of simple computations that may be carried out by dot-selective V4 neurons. Overall, our work demonstrates that the DNN models currently used in computational neuroscience are needlessly large; our approach provides a new way forward for obtaining explainable, high-accuracy models of visual cortical neurons.

show abstract

Section: Discussionmentioning

confidence: 95%

Compact deep neural network models of visual cortex

Cowley,

Stan,

Pillow

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In particular, we found evidence for fluctuating activity in the inferior colliculus (Caruso et al, 2018), primary visual cortex (V1), area V4 (Jun et al, 2022), and the middle fundus (MF) face patch of inferotemporal (IT) cortex (Caruso et al, 2018) when two distinguishable stimuli were presented. We did not observe fluctuating activity in V1 when two stimuli fused to form a common object (i.e., two gratings superimposed to form a "plaid") (Jun et al, 2022).…”

Section: Introductionmentioning

confidence: 93%

“…The experiment involved simultaneous recordings from V1, V4, and MT. As we have previously identified fluctuating response patterns in V1 and V4 (Jun et al, 2022), this MT dataset provides an opportunity to compare the presence of fluctuating activity across visual areas under the same experimental conditions (and some of the same recording sessions) as in our previous work. However, this dataset was comparatively small (as detailed further below), making additional data desirable.…”

Section: Introduction To Datasetsmentioning

confidence: 97%

“…We recently reported a potential solution to this problem: When two stimuli are present, individual neurons may fluctuate between responding to only one or the other stimulus at a time, allowing dynamic populations of neurons to encode different stimuli at the same time. In our earlier work, we developed statistical approaches to evaluate whether the fluctuating activity patterns predicted from such a scheme exist (Caruso et al, 2018;Mohl et al, 2020;Glynn et al, 2021), and we reported evidence for this phenomenon in both auditory (Caruso et al, 2018) and visual (Caruso et al, 2018;Jun et al, 2022) areas. In particular, we found evidence for fluctuating activity in the inferior colliculus (Caruso et al, 2018), primary visual cortex (V1), area V4 (Jun et al, 2022), and the middle fundus (MF) face patch of inferotemporal (IT) cortex (Caruso et al, 2018) when two distinguishable stimuli were presented.…”

Section: Introductionmentioning

confidence: 99%

“…In our earlier work, we developed statistical approaches to evaluate whether the fluctuating activity patterns predicted from such a scheme exist (Caruso et al, 2018;Mohl et al, 2020;Glynn et al, 2021), and we reported evidence for this phenomenon in both auditory (Caruso et al, 2018) and visual (Caruso et al, 2018;Jun et al, 2022) areas. In particular, we found evidence for fluctuating activity in the inferior colliculus (Caruso et al, 2018), primary visual cortex (V1), area V4 (Jun et al, 2022), and the middle fundus (MF) face patch of inferotemporal (IT) cortex (Caruso et al, 2018) when two distinguishable stimuli were presented. We did not observe fluctuating activity in V1 when two stimuli fused to form a common object (i.e., two gratings superimposed to form a "plaid") (Jun et al, 2022).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multiple objects evoke fluctuating responses in several regions of the visual pathway

Schmehl

Caruso

Chen

et al. 2023

Preprint

View full text Add to dashboard Cite

How neural representations preserve information about multiple stimuli is mysterious. Because tuning of individual neurons is coarse (for example, visual receptive field diameters can exceed perceptual resolution), the populations of neurons potentially responsive to each individual stimulus can overlap, raising the question of how information about each item might be segregated and preserved in the population. We recently reported evidence for a potential solution to this problem: when two stimuli were present, some neurons in the macaque visual cortical areas V1 and V4 exhibited fluctuating firing patterns, as if they responded to only one individual stimulus at a time. However, whether such an information encoding strategy is ubiquitous in the visual pathway and thus could constitute a general phenomenon remains unknown. Here we provide new evidence that such fluctuating activity is also evoked by multiple stimuli in visual areas responsible for processing visual motion (middle temporal visual area, MT), and faces (middle fundus and anterolateral face patches in inferotemporal cortex -- areas MF and AL), thus extending the scope of circumstances in which fluctuating activity is observed. Furthermore, consistent with our previous results in the early visual area V1, MT exhibits fluctuations between the representations of two stimuli when these form distinguishable objects but not when they fuse into one perceived object, suggesting that fluctuating activity patterns may underlie visual object formation. Taken together, these findings point toward an updated model of how the brain preserves sensory information about multiple stimuli for subsequent processing and behavioral action.

show abstract

A spatiotemporal mechanism of visual attention: Superdiffusive motion and theta oscillations of neural population activity patterns

Chen

Gong

2022

Sci. Adv.

View full text Add to dashboard Cite

Recent evidence has demonstrated that during visual spatial attention sampling, neural activity and behavioral performance exhibit large fluctuations. To understand the origin of these fluctuations and their functional role, here, we introduce a mechanism based on the dynamical activity pattern (attention spotlight) emerging from neural circuit models in the transition regime between different dynamical states. This attention activity pattern with rich spatiotemporal dynamics flexibly samples from different stimulus locations, explaining many key aspects of temporal fluctuations such as variable theta oscillations of visual spatial attention. Moreover, the mechanism expands our understanding of how visual attention exploits spatially complex fluctuations characterized by superdiffusive motion in space and makes experimentally testable predictions. We further illustrate that attention sampling based on such spatiotemporal fluctuations provides profound functional advantages such as adaptive switching between exploitation and exploration activities and is particularly efficient at sampling natural scenes with multiple salient objects.

show abstract

Coordinated multiplexing of information about separate objects in visual cortex

Cited by 3 publications

References 86 publications

Compact deep neural network models of visual cortex

Compact deep neural network models of visual cortex

Multiple objects evoke fluctuating responses in several regions of the visual pathway

A spatiotemporal mechanism of visual attention: Superdiffusive motion and theta oscillations of neural population activity patterns

Contact Info

Product

Resources

About