Learning to select actions shapes recurrent dynamics in the corticostriatal system

Abstract

Learning to select appropriate actions based on their values is fundamental to adaptive behavior. This form of learning is supported by fronto-striatal systems. The dorsal-lateral prefrontal cortex (dlPFC) and the dorsal striatum (dSTR), which are strongly interconnected, are key nodes in this circuitry. Substantial experimental evidence, including neurophysiological recordings, have shown that neurons in these structures represent key aspects of learning. The computational mechanisms that shape the neurophysiological responses, however, are not clear. To examine this, we developed a recurrent neural network (RNN) model of the dlPFC-dSTR circuit and trained it on an oculomotor sequence learning task. We compared the activity generated by the model to activity recorded from monkey dlPFC and dSTR in the same task. This network consisted of a striatal component which encoded action values, and a prefrontal component which selected appropriate actions. After training, this system was able to autonomously represent and update action values and select actions, thus being able to closely approximate the representational structure in corticostriatal recordings. We found that learning to select the correct actions drove action-sequence representations further apart in activity space, both in the model and in the neural data. The model revealed that learning proceeds by increasing the distance between sequence-specific representations. This makes it more likely that the model will select the appropriate action sequence as learning develops. Our model thus supports the hypothesis that learning in networks drives the neural representations of actions further apart, increasing the probability that the network generates correct actions as learning proceeds. Altogether, this study advances our understanding of how neural circuit dynamics are involved in neural computation, revealing how dynamics in the corticostriatal system support task learning.

Thunderstruck: The ACDC model of flexible sequences and rhythms in recurrent neural circuits

“…In the brain, this orthogonality may be implemented via mixed selectivity of excitatory frontal neurons that ensure downstream readouts without interference [152,153]. Interestingly, Ma ´rton et al [154] recently developed a RNN model of cortico-striatal interactions optimized to learn oculomotor sequences. Similar sequences were performed by awake monkeys while activity was recorded in their dorsolateral prefrontal and striatal areas.…”

Rethinking brain-wide interactions through multi-region ‘network of networks’ models

“…This has given rise to a new class of models-multi-region RNNs (mRNNs)-that describes the activity of many brain regions simultaneously. Such models have been applied to show the necessity of different cortical regions to generate perceptual decisions [25,26], select motor plans [27,28] or actions [29] in response to visual cues, generate robust dynamics for motor output [30], and to study interactions between cortical and subcortical regions [31,32]. The data-driven RNN models described previously can also readily scale to model large, multi-region datasets.…”

Reservoir-based Tracking (TRAKR) For One-shot Classification Of Neural Time-series Patterns

“…That is less likely, however, because the subject was well-trained prior to recordings. Instead, since the signal was recorded over a period of 35 days, the decrease in the classification performance could be a result of degrading signal quality, perhaps due to electrode impedance issues (Kozai et al (2015a;b);Holson et al (1998); Robinson & Camp (1991)).…”

“…A lot of work in computational neuroscience over the past decade has focused on modeling the collective dynamics of a population of neurons in order to gain insight into how firing patterns are related to task variables (Márton et al (2020); Richards et al (2019); ; Remington et al (2018); Kell et al (2018); Zeng et al (2018); Pandarinath et al (2018); Durstewitz (2017); Chaisangmongkon et al (2017); Rajan et al (2016); Sussillo et al (2015); Mante et al (2013); Sussillo & ; ; Sussillo & Abbott (2009)). These approaches, however, rely on fitting the whole dynamical system through many rounds of optimization, either indirectly by modeling the task inputs and outputs (Márton et al (2020); Kell et al (2018); Chaisangmongkon et al (2017); Sussillo et al (2015); Mante et al (2013); , or directly by fitting the weights of a neural network to recorded firing patterns (Pandarinath et al (2018); Durstewitz (2017)). Thus, these approaches can be too time-and computation-intensive for certain applications, e.g.…”