Convolutional recurrent neural network models of dynamics in higher visual cortex

Nayebi, Aran; Kubilius, Jonas; Bear, David M.; Ganguli, Surya; DiCarlo, James J.; Yamins, Daniel

doi:10.1167/18.10.717

Cited by 36 publications

(31 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…properties then what we have documented in a purely feed-forward network. In computational models, recurrent connections are often unfolded into feed-forward layers, effectively making a recurrent model a deeper convolutional model (Nayebi et al, 2018). Although we didn't test deeper architectures in our analysis, we expect that the general principles we described should hold for models with more layers and therefore also for models with recurrent connections.…”

Section: Discussionmentioning

confidence: 99%

Gain, not concomitant changes in spatial receptive field properties, improves task performance in a neural network attention model

Fox

Birman

Gardner

2022

Preprint

View full text Add to dashboard Cite

Attention allows us to focus sensory processing on behaviorally relevant aspects of the visual world. Directing attention has been associated with a number of changes in sensory representation including multiplicative gain as well as shifts in the size and location of neuron receptive fields in early visual cortex. But, which, if any, of these physiological effects can account for the behavioral benefits of attention? Here we use a large scale computational model of primate visual cortex to perform a set of experiments in which we decouple changes in spatial tuning from changes in gain. We show that increased gain at cued locations in a neural network observer model mimics the improvement of human subjects on an attentional task with a spatial cue. Increasing gain resulted in changes in receptive field size and location similar to physiological effects, yet when we forced the model to use only these spatial tuning changes the model failed to produce any behavioral benefit. Instead, we found that gain alone was both necessary and sufficient to explain behavioral improvement during attention. Our results suggest that receptive field shifts are a result of the signal gain that boosts behavioral performance rather than the core mechanism of spatial attention.

show abstract

Section: Discussionmentioning

confidence: 99%

Gain, not concomitant changes in spatial receptive field properties, improves task performance in a neural network attention model

Fox

Birman

Gardner

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…While previous works modeled flexible dynamic learning in biological [1,18] and artificial This work includes a number of limitations. While we enforced several elements of biological fidelity, future work can still be done to further approximate neural network structure to cortical hierarchical specialization, an approach which has been successful in the sensory domain with convolutional neural networks [20,21]. Additionally, this mechanistic approach must be extended beyond the DMS paradigm to more complex, naturalistic tasks, involving high-dimensional state and decision spaces.…”

Section: Discussionmentioning

confidence: 99%

Disinhibitory signaling enables flexible coding of top-down information

Aquino,

Kim,

Rungratsameetaweemana

2023

Preprint

View full text Add to dashboard Cite

Recent studies have proposed employing biologically plausible recurrent neural networks (RNNs) to explore flexible decision making processes in the brain. However, the mechanisms underlying the integration of bottom-up factors (such as incoming sensory signals) and top-down factors (such as task instructions and selective attention) remain poorly understood, both within the context of these models and the brain. To address this question, we trained biologically inspired RNNs on complex cognitive tasks that require adaptive integration of these factors. By performing extensive dynamical systems analyses, we show that our RNN model is capable of seamlessly incorporating top-down signals with sensory signals to perform the complex tasks. Furthermore, through comprehensive local connectivity analyses, we identified important inhibitory feedback signals that efficiently modulate the bottom-up sensory coding in a task-driven manner. Finally, we introduced an anatomical constraint where a specific subgroup of neurons receives the sensory input signal, effectively creating a designated sensory area within the RNN. Through this constraint, we show that these "sensory" neurons possess the remarkable ability to multiplex and dynamically combine both bottom-up and top-down information. These findings are consistent with recent experimental results highlighting that such integration is a key factor in facilitating flexible decision making. Overall, our work provides a framework for generating testable hypotheses for the hierarchical encoding of task-relevant information.

show abstract

“…Physiological recordings have uncovered several features of the processing in the human visual system that are relevant to judging the plausibility of the networks examined here. First, the conduction from one area to another in the visual cortex (roughly corresponding to different layers in neural networks) takes approximately 10 ms 48 , with signal from the photoreceptors reaching the top of the visual hierarchy in inferior temporal cortex in 70-100 ms 49 . Therefore, a single sweep from input to output in a purely feedforward network should result in decisions with RT less than a few hundred milliseconds even though human decisions can range from a hundred of milliseconds to a few seconds.…”

Section: Discussionmentioning

confidence: 99%

“…Nevertheless, one critical limitation of the biological plausibility of RTNet is its lack of recurrency. That being said, the question of how to train recurrent neural networks on static images remains open 44, 49, 58–60 . Further, while the core of RTNet does not include recurrency, the evidence accumulation system can be thought of as a recurrent network.…”

Section: Discussionmentioning

confidence: 99%

RTNet neural network exhibits the signatures of human perceptual decision making

Rafiei

Rahnev

2022

Preprint

View full text Add to dashboard Cite

Convolutional neural networks currently provide the best models of biological vision. However, their decision behavior, including the facts that they are deterministic and use equal number of computations for easy and difficult stimuli, differs markedly from human decision-making, thus limiting their applicability as models of human perceptual behavior. Here we develop a new neural network, RTNet, that generates stochastic decisions and human-like response time (RT) distributions, and also reproduces all foundational features of human accuracy, RT, and confidence. To test RTNet's ability to predict human behavior on novel images, we collected accuracy, RT, and confidence data from 60 human subjects performing a digit discrimination task. We found that the accuracy, RT, and confidence produced by RTNet for individual novel images correlated with the same quantities produced by human subjects. Critically, human subjects who were more similar to the average human performance were also found to be closer to RTNet's predictions. Overall, RTNet is the first neural network that exhibits all basic signatures of perceptual decision making, and therefore provides the most detailed model of all critical features of human behavior for novel images.

show abstract

Convolutional recurrent neural network models of dynamics in higher visual cortex

Cited by 36 publications

References 0 publications

Gain, not concomitant changes in spatial receptive field properties, improves task performance in a neural network attention model

Gain, not concomitant changes in spatial receptive field properties, improves task performance in a neural network attention model

Disinhibitory signaling enables flexible coding of top-down information

RTNet neural network exhibits the signatures of human perceptual decision making

Contact Info

Product

Resources

About