Controllability Governs the Balance Between Pavlovian and Instrumental Action Selection

Dorfman, Hayley M.; Gershman, Samuel J.

doi:10.1101/596577

Cited by 9 publications

(21 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [18], a Bayesian framework was introduced that formalized action valuation in terms of probabilistic inference ( Fig 1B). According to this framework, Pavlovian and instrumental processes correspond to distinct predictive models of reward (or punishment) outcomes.…”

Section: Modeling and Behavioral Resultsmentioning

confidence: 99%

“…The Pavlovian weight can also be interpreted as the subjective degree of belief in an uncontrollable environment, where actions do not influence the probability distribution over outcomes (and correspondingly, 1 − w is the degree of belief in a controllable environment). Dorfman and Gershman [18] compared two versions of probabilistic arbitration. In the Fixed Bayesian model, the Pavlovian weight reflects a priori beliefs (i.e., prior to observing data).…”

Section: Modeling and Behavioral Resultsmentioning

confidence: 99%

“…Accordingly, Pavlovian bias increases with the amount of Pavlovian training [19]. Dorfman and Gershman [18] tested the controllability prediction more directly using a variant of the Go/NoGo paradigm, which has been widely employed as an assay of Pavlovian bias in human subjects [22,23,24,21,20,25]. This task crosses valence (winning reward vs. avoiding punishment) with action (Go vs. NoGo), resulting in four conditions: Go- [20] in their EEG study, which differed in several minor ways from the design used by [21] in their fMRI study (see Materials and Methods).…”

Section: Introductionmentioning

confidence: 99%

“…This approach has proven successful in understanding arbitration between different instrumental learning processes [14,15,16,17]. More recently, it has been used to understand arbitration between Pavlovian and instrumental processes [18]. The key idea is that instrumental learning is more statistically flexible, in the sense that it can learn reward predictions that are both action-specific and stimulus-specific, whereas Pavlovian learning can only learn stimulusspecific predictions.…”

Section: Introductionmentioning

confidence: 99%

“…The account sketched above can be formalized in terms of Bayesian model comparison [18]. Several implications follow.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Neural signatures of arbitration between Pavlovian and instrumental action selection

Gershman

Guitart-Masip

Cavanagh

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Pavlovian associations drive approach towards reward-predictive cues, and avoidance of punishment-predictive cues. These associations "misbehave" when they conflict with correct instrumental behavior. This raises the question of how Pavlovian and instrumental influences on behavior are arbitrated. We test a computational theory according to which Pavlovian influence will be stronger when inferred controllability of outcomes is low. Using a model-based analysis of a Go/NoGo task with human subjects, we show that theta-band oscillatory power in frontal cortex tracks inferred controllability, and that these inferences predict Pavlovian action biases. Functional MRI data revealed an inferior frontal gyrus correlate of action probability and a ventromedial prefrontal correlate of outcome valence, both of which were modulated by inferred controllability.

show abstract

Section: Modeling and Behavioral Resultsmentioning

confidence: 99%

Section: Modeling and Behavioral Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…The account sketched above can be formalized in terms of Bayesian model comparison [18]. Several implications follow.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Neural signatures of arbitration between Pavlovian and instrumental action selection

Gershman

Guitart-Masip

Cavanagh

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Chemistry of the Adaptive Mind: Lessons from Dopamine

Cools

2019

Neuron

115

114

View full text Add to dashboard Cite

The brain faces various computational tradeoffs, such as the stability-flexibility dilemma. The major ascending neuromodulatory systems are well suited to dynamically regulate these tradeoffs depending on changing task demands. This follows from various general principles of chemical neuromodulation, which are illustrated with evidence from pharmacological neuroimaging studies on striatal dopamine's role in output gating and cost-benefit choice of cognitive tasks. The work raises open questions, including those regarding the top-down cortical control of the midbrain dopamine system, and begins to elucidate the mechanisms underlying the variability in catecholaminergic drug effects. Such drug effects depend on the baseline state of distinct target brain regions, reflecting, in part, the systems' self-regulatory capacity to maintain equilibrium. It is hypothesized that the basal tone of different dopaminergic projection systems reflects the perceived statistics of the environment computed in frontal cortex. By normalizing dopamine levels, dopaminergic drugs might counteract the bias elicited by the perceived environment.

show abstract

Stress-sensitive inference of task controllability

Ligneul

Mainen

et al. 2020

Preprint

View full text Add to dashboard Cite

Estimating environmental controllability enables agents to better predict upcoming events and decide when to engage controlled action selection. How does the human brain estimate environmental controllability? Trial-by-trial analysis of choices, decision times and neural activity in an explore-and-predict task demonstrate that humans solve this problem by comparing the predictions of an “actor” model with those of a reduced “spectator” model of their environment. Neural BOLD responses within striatal and medial prefrontal areas tracked the instantaneous difference in the prediction errors generated by these two statistical learning models. BOLD activity in the posterior cingulate, parietal and prefrontal cortices covaried with changes in estimated controllability. Exposure to inescapable stressors biased controllability estimates downward and increased reliance on the spectator model in an anxiety-dependent fashion. Taken together, these findings provide a mechanistic account of controllability inference and its distortion by stress exposure.

show abstract

Controllability Governs the Balance Between Pavlovian and Instrumental Action Selection

Cited by 9 publications

References 28 publications

Neural signatures of arbitration between Pavlovian and instrumental action selection

Neural signatures of arbitration between Pavlovian and instrumental action selection

Chemistry of the Adaptive Mind: Lessons from Dopamine

Stress-sensitive inference of task controllability

Contact Info

Product

Resources

About