The visual system of mammals is comprised of parallel, hierarchical specialized pathways. Different pathways are specialized in so far as they use representations that are more suitable for supporting specific downstream behaviours. In particular, the clearest example is the specialization of the ventral ('what') and dorsal ('where') pathways of the visual cortex. These two pathways support behaviours related to visual recognition and movement, respectively. To-date, deep neural networks have mostly been used as models of the ventral, recognition pathway. However, it is unknown whether both pathways can be modelled with a single deep ANN. Here, we ask whether a single model with a single loss function can capture the properties of both the ventral and the dorsal pathways. We explore this question using data from mice, who like other mammals, have specialized pathways that appear to support recognition and movement behaviours. We show that when we train a deep neural network architecture with two parallel pathways using a self-supervised predictive loss function, we can outperform other models in fitting mouse visual cortex. Moreover, we can model both the dorsal and ventral pathways. These results demonstrate that a self-supervised predictive learning approach applied to parallel pathway architectures can account for some of the functional specialization seen in mammalian visual systems.

Neurons in the dorsal visual pathway of the mammalian brain are selective for motion stimuli, with the complexity of stimulus representations increasing along the hierarchy. This progression is similar to that of the ventral visual pathway, which is well characterized by artificial neural networks (ANNs) optimized for object recognition. In contrast, there are no image-computable models of the dorsal stream with comparable explanatory power. We hypothesized that the properties of dorsal stream neurons could be explained by a simple learning objective: the need for an organism to orient itself during self-motion. To test this hypothesis, we trained a 3D ResNet in a self-supervised manner to predict an agent’s self-motion parameters from visual stimuli in a simulated environment. We found that the responses in this network accounted well for the selectivity of neurons in a large database of single-neuron recordings from the dorsal visual stream of non-human primates. In contrast, ANNs trained for action recognition or with a contrastive objective could not explain responses in the dorsal stream, despite also being trained on naturalistic videos with moving objects. These results demonstrate that an ecologically relevant, self-supervised cost function can account for dorsal stream properties in the primate brain.

, christopher c. pack & Abbas f. Sadikot * inhibiting inappropriate actions in a context is an important part of the human cognitive repertoire, and deficiencies in this ability are common in neurological and psychiatric disorders. An anti-saccade is a simple oculomotor task that tests this ability by requiring inhibition of saccades to peripheral targets (pro-saccade) and producing voluntary eye movements toward the mirror position (anti-saccades). Previous studies provide evidence for a possible contribution from the basal ganglia in anti-saccade behavior, but the precise role of different components is still unclear. Parkinson's disease patients with implanted deep brain stimulators (DBS) in subthalamic nucleus (STN) provide a unique opportunity to investigate the role of the STN in anti-saccade behavior. Previous attempts to show the effect of STN DBS on anti-saccades have produced conflicting observations. For example, the effect of STN DBS on anti-saccade error rate is not yet clear. Part of this inconsistency may be related to differences in dopaminergic states in different studies. Here, we tested Parkinson's disease patients on anti-and pro-saccade tasks ON and OFF STN DBS, in ON and OFF dopaminergic medication states. First, STN DBS increases anti-saccade error rate while patients are OFF dopamine replacement therapy. Second, dopamine replacement therapy and STN DBS interact: L-dopa reduces the effect of STN DBS on antisaccade error rate. Third, STN DBS induces different effects on pro-and anti-saccades in different patients. these observations provide evidence for an important role for the Stn in the circuitry underlying context-dependent modulation of visuomotor action selection. Neuromodulation or Deep Brain Stimulation (DBS) of several subcortical areas significantly improves motor function in Parkinson's disease (PD) 1,2. Stimulation of the subthalamic nucleus (STN) improves tremor, drug-induced dyskinesias and motor fluctuations, and may also modulate aspects of executive cognition or emotion 3-6. However, unlike its motor outcome, the effect of subthalamic DBS on higher order executive functions remains uncertain. An important component of executive function is inhibitory control: the ability to suppress an impulsive response to external events due to an inappropriate context 7. Dysfunction of inhibitory control is involved in several psychiatric conditions, such as mood disorders, disorders of thought, and addiction 8,9. In PD, a deficiency of inhibitory control has been reported in different experimental tasks, such as the stop signal task 10 , and the Stroop task 11 , and impulse control may be worsened by therapies in some patients 12,13. Eye movements can be measured precisely and objectively (with 1 ms temporal resolution and approximately 0.5 degree spatial resolution) by using state-of-the-art eye-tracking technologies. This advance has made eye movements a popular readout behavior for probing different aspects of cognition with potential applications in clinical assessments 14. One well-...