A One-Shot Shift from Explore to Exploit in Monkey Prefrontal Cortex

Achterberg, Jascha; Kadohisa, Mikiko; Watanabe, Kei; Kusunoki, Makoto; Buckley, Mark J.; Duncan, John

doi:10.1523/jneurosci.1338-21.2021

Cited by 7 publications

(12 citation statements)

References 61 publications

(80 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using fMRI, we investigated the neural correlates of the assessment of the possibility to use the information collected during the choice in the future, manipulated through horizon length, as well as the assessment of the contingency between choice and information, manipulated through the availability of the counterfactual outcome. Modulation of activity associated with exploratory behavior in an uncertain environment has been recorded in humans and monkeys in both the ACC and the MCC [42,[52][53][54], but here, we found interesting anatomical distinctions. We found that the pgACC was more active when the information could be used in the future in the long horizon.…”

Section: Strategic Exploration Signals In Acc/mcc and Dlpfcsupporting

confidence: 51%

“…Specifically, a network consisting of the MCC, the dlPFC, and, potentially, the locus coeruleus could support the relaxation of the effect of expected value on choices based on the context. Altogether, these results illustrate how ACC/MCC and dlPFC might dynamically switch modes to pursue different goals depending on the task demands [ 42 , 52 , 54 ]. Future studies will aim at testing whether switching mode is dependent on noradrenergic inputs and which causal role both regions play in changing into and out of strategic exploration.…”

Section: Discussionmentioning

confidence: 96%

See 1 more Smart Citation

Neural responses in macaque prefrontal cortex are linked to strategic exploration

et al. 2023

View full text Add to dashboard Cite

Humans have been shown to strategically explore. They can identify situations in which gathering information about distant and uncertain options is beneficial for the future. Because primates rely on scarce resources when they forage, they are also thought to strategically explore, but whether they use the same strategies as humans and the neural bases of strategic exploration in monkeys are largely unknown. We designed a sequential choice task to investigate whether monkeys mobilize strategic exploration based on whether that information can improve subsequent choice, but also to ask the novel question about whether monkeys adjust their exploratory choices based on the contingency between choice and information, by sometimes providing the counterfactual feedback about the unchosen option. We show that monkeys decreased their reliance on expected value when exploration could be beneficial, but this was not mediated by changes in the effect of uncertainty on choices. We found strategic exploratory signals in anterior and mid-cingulate cortex (ACC/MCC) and dorsolateral prefrontal cortex (dlPFC). This network was most active when a low value option was chosen, which suggests a role in counteracting expected value signals, when exploration away from value should to be considered. Such strategic exploration was abolished when the counterfactual feedback was available. Learning from counterfactual outcome was associated with the recruitment of a different circuit centered on the medial orbitofrontal cortex (OFC), where we showed that monkeys represent chosen and unchosen reward prediction errors. Overall, our study shows how ACC/MCC-dlPFC and OFC circuits together could support exploitation of available information to the fullest and drive behavior towards finding more information through exploration when it is beneficial.

show abstract

Section: Strategic Exploration Signals In Acc/mcc and Dlpfcsupporting

confidence: 51%

Section: Discussionmentioning

confidence: 96%

Neural responses in macaque prefrontal cortex are linked to strategic exploration

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Using fMRI, we investigated the neural correlates of the assessment of the possibility to use the information collected during the choice in the future, manipulated through horizon length, as well as the assessment of the contingency between choice and information, manipulated through the availability of the counterfactual outcome. Modulation of activity associated with exploratory behavior in an uncertain environment has been recorded in humans and monkeys in both the ACC and the MCC ( 5, 24, 39, 51–53 ) but here we found interesting anatomical distinctions. We found that the pgACC was more active when the information could be used in the future in the long horizon.…”

Section: Discussionsupporting

confidence: 44%

“…Specifically, a network consisting of the MCC, the dlPFC and potentially the locus coeruleus could support the relaxation of the effect of expected value on choices based on the context. Altogether, these results illustrate how ACC/MCC and dlPFC might dynamically switch modes to pursue different goals depending on the task demands (24,39,53). Future studies will aim at testing whether switching mode is dependent on noradrenergic inputs and which causal role both regions play in changing into and out of strategic exploration.…”

Section: Strategic Exploration Signals In Acc/mcc and Dlpfcmentioning

confidence: 77%

Strategic exploration in the macaque’s prefrontal cortex

Jahn

Grohn

Cuell

et al. 2022

Preprint

View full text Add to dashboard Cite

Humans have been shown to strategically explore. They can identify situations in which gathering information about distant and uncertain options is beneficial for the future. Because primates rely on scare resources when they forage, they are also thought to strategically explore, but whether they use the same strategies as humans and the neural bases of strategic exploration in monkeys are largely unknown. We designed a sequential choice task to investigate whether monkeys mobilize strategic exploration based on whether that information can improve subsequent choice, but also to ask the novel question about whether monkeys adjust their exploratory choices based on the contingency between choice and information, by sometimes providing the counterfactual feedback, about the option not chosen. We show that monkeys decreased their reliance on expected value when exploration could be beneficial, but this was not mediated by changes in the effect of uncertainty on choices. We found strategic exploratory signals in anterior and mid-cingulate cortex (ACC/MCC) and dorsolateral prefrontal cortex (dlPFC). This network was most active when a low value option was chosen which suggests a role in counteracting expected value signals, when exploration away from value should to be considered. Such strategic exploration was abolished when the counterfactual feedback was available. Learning from counterfactual outcome was associated with the recruitment of a different circuit centered on the medial orbitofrontal cortex (OFC), where we showed that monkeys represent chosen and unchosen reward prediction errors. Overall, our study shows how ACC/MCC-dlPFC and OFC circuits together could support exploitation of available information to the fullest and drive behavior towards finding more information through exploration when it is beneficial.

show abstract

“…Optimal ‘one-shot’ transitions would require only a single error trial to successfully switch behaviour 23,24 , as seen in Fig. 1d.…”

Section: Mainmentioning

confidence: 99%

Prediction-error signals in anterior cingulate cortex drive task-switching

Cole

Harvey

Myers-Joseph

et al. 2022

Preprint

View full text Add to dashboard Cite

Task-switching is a fundamental cognitive ability which requires animals to update their knowledge of current rules, allowing flexible behaviour in a changing environment1. This is often achieved through evaluating discrepancies between observed and expected events. The anterior cingulate cortex (ACC) has a key role in processing such discrepancies, or prediction errors2–12. However, the neural circuit mechanisms underlying task-switching are largely unknown. Here we show that activity in the ACC induced by the absence of expected stimuli is necessary for rapid task-switching. Mice trained to perform a block-wise set-shifting task typically required a single experience of an expectation violation, or prediction error to accurately switch between responding to the same stimuli using distinct rules. Neurons in the ACC explicitly represented these prediction errors, and their activity was predictive of successful one-shot behavioural transitions. Prediction error signals were projection targetspecific, constrained in their spatio-temporal spread across cortex, and heavily disrupted by VIP interneuron perturbation. Optogenetic silencing and single-trial un-silencing revealed that the requirement of the ACC in task-switching was restricted to the epochs when neural prediction error signals were observed. These results reveal a dedicated circuitry promoting the transition between distinct cognitive states.

show abstract

A One-Shot Shift from Explore to Exploit in Monkey Prefrontal Cortex

Cited by 7 publications

References 61 publications

Neural responses in macaque prefrontal cortex are linked to strategic exploration

Neural responses in macaque prefrontal cortex are linked to strategic exploration

Strategic exploration in the macaque’s prefrontal cortex

Prediction-error signals in anterior cingulate cortex drive task-switching

Contact Info

Product

Resources

About