Corticostriatal synaptic weight evolution in a two-alternative forced choice task: a computational study

Vich, Catalina; Dunovan, Kyle; Verstynen, Timothy D.; Rubin, Jonathan E.

doi:10.1016/j.cnsns.2019.105048

Cited by 15 publications

(38 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, recent optogenetic work in mice suggests that activating the subthalamic nucleus, a key node in the indirect pathway, not only halts the motoric response but also interrupts cognitive processes related to action selection (Heston et al, 2020). Our current observations, combined with this previous work, suggests that the decision policy reconfiguration that we observe may associate with similar underlying corticostriatal dynamics, with belief-driven changes to drift rate varying with the difference in direct pathway firing rates across action channels Dunovan et al 2019, and change-point-probability-driven changes to the boundary height varying with overall indirect pathway activity Vich et al, 2020). Future physiological studies should focus on validating this predicted relationship between decision policy reconfiguration and CBGT pathways.…”

Section: Discussionsupporting

confidence: 81%

“…More explicitly, both directed and random exploration have been linked to variations in genes that affect dopamine levels in prefrontal cortex and striatum, respectively ( Gershman and Tzovaras, 2018) . Physiologically, previous work has found that a dopamine-controlled spike-timing dependent plasticity rule alters the ratio of direct to indirect pathway efficacy in a simulated corticostriatal network ( Vich et al, 2020 ), with overall indirect pathway activity linked to the modulation of the boundary height in a DDM and the difference in direct pathway activation across action channels associating with changes in the drift rate ( Dunovan et al, 2019 ; Rubin et al, 2020 ). Moreover, recent optogenetic work in mice suggests that activating the subthalamic nucleus, a key node in the indirect pathway, not only halts the motoric response but also interrupts cognitive processes related to action selection ( Heston et al, 2020 ).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Dynamic decision policy reconfiguration under outcome uncertainty

Bond

Dunovan

Porter³

et al. 2020

Preprint

View full text Add to dashboard Cite

Humans and other mammals flexibly select actions under noisy and unstable conditions. To shed light on the mechanism driving this flexibility, we evaluated how the underlying decision policy evolves when humans change their minds about the most rewarding action. Participants performed a dynamic variant of the two-armed bandit task that manipulated the certainty in relative reward probabilities (conflict) and the reliability of action-outcome contingencies (volatility). We found that conflict and volatility contributed to shifts in exploratory states by changing both the rate of evidence accumulation (drift rate) and the amount of evidence needed to make a decision (boundary height). Following a switch in the optimal choice, the drift rate and the boundary height reduce, allowing variability in the accumulation process to predominate action selection, leading to a fast exploratory state. These changes facilitate the discovery of the new optimal choice, with a quick recovery of the boundary height to baseline. In parallel, the drift rate gradually returns to its asymptotic value as the belief in the value of the optimal choice stabilizes. Together, these decision dynamics suggest that, in the context of volatile two-choice decisions, humans adopt a combined information-threshold and drift rate mechanism in response to environmental changes. Unlike previous observations, we found no evidence that fluctuations in norepinephrine, as measured by pupillometry, associated with this adaptive shift toward an exploratory policy. We conclude that the multifaceted processes underlying a decision can rapidly reconfigure to adapt action selection policy under multiple forms of environmental uncertainty.

show abstract

Section: Discussionsupporting

confidence: 81%

Section: Discussionmentioning

confidence: 99%

Dynamic decision policy reconfiguration under outcome uncertainty

Bond

Dunovan

Porter³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Algorithm 1 Normative form of the plasticity algorithm for corticostriatal synapses. See Vich et al (2020) for a specific implementation.…”

mentioning

confidence: 99%

The credit assignment problem in cortico‐basal ganglia‐thalamic networks: A review, a problem and a possible solution

Rubin

Vich

Clapp

et al. 2020

Eur J of Neuroscience

Self Cite

View full text Add to dashboard Cite

The question of how cortico-basal ganglia-thalamic (CBGT) pathways use dopaminergic feedback signals to modify future decisions has challenged computational neuroscientists for decades. Reviewing the literature on computational representations of dopaminergic corticostriatal plasticity, we show how the field is converging on a normative, synaptic-level learning algorithm that elegantly captures both neurophysiological properties of CBGT circuits and behavioral dynamics during reinforcement learning. Unfortunately, the computational studies that have led to this normative algorithmic model have all relied on simplified circuits that use abstracted actionselection rules. As a result, the application of this corticostriatal plasticity algorithm to a full model of the CBGT pathways immediately fails because the spatiotemporal distance between integration (corticostriatal circuits), action selection (thalamocortical loops) and learning (nigrostriatal circuits) means that the network does not know which synapses should be reinforced to favor previously rewarding actions. We show how observations from neurophysiology, in particular the sustained activation of selected action representations, can provide a simple means of resolving this credit assignment problem in models of CBGT learning. Using a biologically realistic spiking model of the full CBGT circuit, we demonstrate how this solution can allow a network to learn to select optimal targets and to relearn action-outcome contingencies when the environment changes. This simple illustration highlights how the normative framework for corticostriatal plasticity can be expanded to capture macroscopic network dynamics during learning and decision-making. | 2235RUBIN et al.

show abstract

“…In contrast, negative RPEs have the opposite effect, enhancing the sensitivity the indirect pathway while depressing those of the direct pathway ( Gurney et al., 2015 ). This opposing plasticity effect means that gains reinforce the appropriately selected action over less rewarding alternatives and push the network into a more exploitative state; while losses reduce the saliency of the selected action which allows for more competition between the action channels, pushing the network into a more exploratory state ( Cools et al., 2009 ; Stauffer et al., 2014 ; Collins and Frank, 2015 ; Dunovan et al., 2019 ; Vich et al., 2020 ) Because the drift rate of individual action decisions reflects competition between the direct and indirect pathways within an action channel ( Dunovan and Verstynen, 2016 ; Dunovan et al., 2019 ), we used positive and negative RPEs to modulate the relative Go and No Go drift rates of each action channel independently ( Dunovan et al., 2019 )…”

Section: Methodsmentioning

confidence: 99%

“…During learning, phasic dopamine responses, thought to reflect reward prediction errors ( Schultz et al., 1997 ), have opposing influences on the direct and indirect pathways, depending on the action that they represent and the nature of the feedback signal (i.e. gain or loss) ( Bogacz and Larsen, 2011 ; Gurney et al., 2015 ; Vich et al., 2020 ). Specifically, if direct pathway neurons, with D1 receptors, are eligible for plasticity when a phasic dopamine response occurs (i.e.…”

Section: Introductionmentioning

confidence: 99%

Adiposity covaries with signatures of asymmetric feedback learning during adaptive decisions

Verstynen

Dunovan

Walsh

et al. 2020

Social Cognitive and Affective Neuroscience

View full text Add to dashboard Cite

Abstract Unhealthy weight gain relates, in part, to how people make decisions based on prior experience. Here we conducted post hoc analysis on an archival data set to evaluate whether individual differences in adiposity, an anthropometric construct encompassing a spectrum of body types, from lean to obese, associate with signatures of asymmetric feedback learning during value-based decision-making. In a sample of neurologically healthy adults (N = 433), ventral striatal responses to rewards, measured using fMRI, were not directly associated with adiposity, but rather moderated its relationship with feedback-driven learning in the Iowa Gambling Task tested outside the scanner. Using a biologically-inspired model of basal ganglia-dependent decision processes, we found this moderating effect of reward reactivity to be explained by an asymmetrical use of feedback to drive learning; that is, with more plasticity for gains than for losses, stronger reward reactivity leads to decisions that minimize exploration for maximizing long-term outcomes. Follow-up analysis confirmed that individual differences in adiposity correlated with signatures of asymmetric use of feedback cues during learning, suggesting that reward reactivity may especially relate to adiposity, and possibly obesity risk, when gains impact future decisions more than losses.

show abstract

Corticostriatal synaptic weight evolution in a two-alternative forced choice task: a computational study

Cited by 15 publications

References 52 publications

Dynamic decision policy reconfiguration under outcome uncertainty

Dynamic decision policy reconfiguration under outcome uncertainty

The credit assignment problem in cortico‐basal ganglia‐thalamic networks: A review, a problem and a possible solution

Adiposity covaries with signatures of asymmetric feedback learning during adaptive decisions

Contact Info

Product

Resources

About