Cortico-basal-ganglia-thalamic (CBGT) networks are critical for adaptive decision-making, yet how changes to circuit-level properties impact cognitive algorithms remains unclear. Here we explore how dopaminergic plasticity at corticostriatal synapses alters competition between striatal pathways, impacting the evidence accumulation process during decision-making. Spike-timing dependent plasticity simulations showed that dopaminergic feedback based on rewards modified the ratio of direct and indirect corticostriatal weights within opposing action channels. Using the learned weight ratios in a full spiking CBGT network model, we simulated neural dynamics and decision outcomes in a reward-driven decision task and fit them with a drift diffusion model. Fits revealed that the rate of evidence accumulation varied with inter-channel differences in direct pathway activity while boundary height varied with overall indirect pathway activity. This multi-level modeling approach demonstrates how complementary learning and decision computations can emerge from corticostriatal plasticity.

The question of how cortico-basal ganglia-thalamic (CBGT) pathways use dopaminergic feedback signals to modify future decisions has challenged computational neuroscientists for decades. Reviewing the literature on computational representations of dopaminergic corticostriatal plasticity, we show how the field is converging on a normative, synaptic-level learning algorithm that elegantly captures both neurophysiological properties of CBGT circuits and behavioral dynamics during reinforcement learning. Unfortunately, the computational studies that have led to this normative algorithmic model have all relied on simplified circuits that use abstracted actionselection rules. As a result, the application of this corticostriatal plasticity algorithm to a full model of the CBGT pathways immediately fails because the spatiotemporal distance between integration (corticostriatal circuits), action selection (thalamocortical loops) and learning (nigrostriatal circuits) means that the network does not know which synapses should be reinforced to favor previously rewarding actions. We show how observations from neurophysiology, in particular the sustained activation of selected action representations, can provide a simple means of resolving this credit assignment problem in models of CBGT learning. Using a biologically realistic spiking model of the full CBGT circuit, we demonstrate how this solution can allow a network to learn to select optimal targets and to relearn action-outcome contingencies when the environment changes. This simple illustration highlights how the normative framework for corticostriatal plasticity can be expanded to capture macroscopic network dynamics during learning and decision-making. | 2235RUBIN et al.

No abstract

Cortico-basal-ganglia-thalamic (CBGT) networks are critical for adaptive decision-making, yet how changes to circuit-level properties impact cognitive algorithms remains unclear. Here we explore how dopaminergic plasticity at corticostriatal synapses alters competition between striatal pathways, impacting the evidence accumulation process during decision-making. Spike-timing dependent plasticity simulations showed that dopaminergic feedback based on rewards modified the ratio of direct and indirect corticostriatal weights within opposing action channels. Using the learned weight ratios in a full spiking CBGT network model, we simulated neural dynamics and decision outcomes in a reward-driven decision task and fit them with a drift diffusion model. Fits revealed that the rate of evidence accumulation varied with inter-channel differences in direct pathway activity while boundary height varied with overall indirect pathway activity. This multi-level modeling approach demonstrates how complementary learning and decision computations can emerge from corticostriatal plasticity. PLOS1/34 choice towards more rewarding actions. By mapping "up" the levels of analysis, this approach yields specific predictions about aspects of neuronal activity that map to the quantities appearing in the cognitive decision-making framework. 1 Introduction 1 The flexibility of mammalian behavior showcases the dynamic range over which 2 neural circuits can be modified by experience and the robustness of the emergent 3 cognitive algorithms that guide goal-directed actions. Decades of research in cognitive 4 science has independently detailed the algorithms of decision-making (e.g., 5 accumulation-to-bound models, [1]) and reinforcement learning (RL; [2, 3]), providing 6 foundational insights into the computational principles of adaptive decision-making. In 7 parallel, research in neuroscience has shown how the selection of actions, and the use of 8 feedback to modify selection processes, both rely on a common neural substrate: 9 cortico-basal ganglia-thalamic (CBGT) circuits [4-8]. 10 42 speed at which evidence accumulates in favor of one choice over another (see [20]). It 43 remains unclear how the dual-pathway organization of corticostriatal inputs contributes 44 to the representation and comparison of evidence for conflicting actions. Recent 45 theoretical models have proposed that decision evidence may be encoded as a dynamic 46 competition between the direct and indirect pathways within a single CBGT action 47 PLOS 2/34 57 behavior across neural and cognitive levels of analysis (Figure 1). Using a preliminary 58 spike-timing dependent plasticity (STDP; [24, 25]) simulation, we modeled how phasic 59 DA feedback signals [26] can modulate the relative balance of corticostriatal synapses, 60 thereby promoting or deterring action selection. The effects of learning on the synaptic 61 weights were incorporated into a spiking model of the full CBGT network meant to 62 capture the known physiological properties and connectivity patterns of the co...

No abstract