Memory and decision making interact to shape the value of unchosen options

Biderman, Natalie; Shohamy, Daphna

doi:10.1038/s41467-021-24907-x

Cited by 29 publications

(38 citation statements)

References 75 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, transfer phase choices could almost perfectly be reproduced from explicit ratings, which were, in turn, more consistent with range, rather than divisive normalization process. In addition to provide a welcome test of robustness of our results, the similarity between choicebased and rating-based subjective values, also demonstrates the context-dependent valuation span across procedural as well as declarative representational systems [51,52].…”

Section: Discussionmentioning

confidence: 99%

The functional form of value normalization in human reinforcement learning

Bavard

Palminteri

2022

Preprint

View full text Add to dashboard Cite

Reinforcement learning research in humans and other species indicates that rewards are represented in a context-dependent manner. More specifically, reward representations seem to be normalized as a function of the value of the alternative options. The dominant view postulates that value context-dependence is achieved via a divisive normalization rule, inspired by perceptual decision-making research. However, behavioral and neural evidence points to another plausible mechanism: range normalization. Critically, previous experimental designs were ill-suited to disentangle the divisive and the range normalization accounts, which generate similar behavioral predictions in many circumstances. To address this question, we designed a new learning task where we manipulated, across learning contexts, the number of options and the value ranges. Behavioral and computational analyses falsify the divisive normalization account and rather provide support for the range normalization rule. Together, these results shed new light on the computational mechanisms underlying context-dependence in learning and decision-making.

show abstract

Section: Discussionmentioning

confidence: 99%

The functional form of value normalization in human reinforcement learning

Bavard

Palminteri

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Episodic memory of previous choices on stimulus affects current choice in a learning task (Bornstein & Norman, 2017). Counterfactual learning of items chosen against one another is modulated by the strength of the episodic memory for them (Biderman & Shohamy, 2021). In some people, memory strength and RL learning rate seem to trade off depending on experimental learning context (Yifrah et al, 2021).…”

Section: Interactions With Other Processesmentioning

confidence: 99%

How Working Memory and Reinforcement Learning Are Intertwined: A Cognitive, Neural, and Computational Perspective

Yoo

Collins

2022

Journal of Cognitive Neuroscience

View full text Add to dashboard Cite

Reinforcement learning and working memory are two core processes of human cognition and are often considered cognitively, neuroscientifically, and algorithmically distinct. Here, we show that the brain networks that support them actually overlap significantly and that they are less distinct cognitive processes than often assumed. We review literature demonstrating the benefits of considering each process to explain properties of the other and highlight recent work investigating their more complex interactions. We discuss how future research in both computational and cognitive sciences can benefit from one another, suggesting that a key missing piece for artificial agents to learn to behave with more human-like efficiency is taking working memory's role in learning seriously. This review highlights the risks of neglecting the interplay between different processes when studying human behavior (in particular when considering individual differences). We emphasize the importance of investigating these dynamics to build a comprehensive understanding of human cognition.

show abstract

“…Participants who guessed the answers were more curious about the actual answer than those in the passive condition and showed a larger pupil dilation response in anticipation of the correct answer. It is worth adding that, when expectations about the chosen outcome are not met, uncertainty about the unchosen outcome may also increase, leading to increased curiosity (see [67] for evidence that the learned value of a chosen option can affect the inferred value of an unchosen option). This may explain why people tend to spontaneously engage in counterfactual reasoning after unexpected outcomes [68], and why children selectively seek counterfactual information about alternatives that they could have chosen [5].…”

Section: Motivationally Relevant Characteristics Of Counterfactual In...mentioning

confidence: 99%

Counterfactual curiosity: motivated thinking about what might have been

FitzGibbon

Murayama

2022

Phil. Trans. R. Soc. B

View full text Add to dashboard Cite

Counterfactual information, information about what might have been, forms the content of counterfactual thoughts and emotions like regret and relief. Recent research suggests that human adults and children, as well as rhesus monkeys, demonstrate ‘counterfactual curiosity’: they are motivated to seek out counterfactual information after making decisions. Based on contemporary theories of curiosity and information seeking and a broad range of empirical literature, we suggest multiple heterogeneous psychological processes that contribute to people's motivation for counterfactual information. This includes processes that are identified in the curiosity literature more generally—the potential use of counterfactual information for adaptive decision making (its long-term instrumental value) and the drive to reduce uncertainty. Additionally, we suggest that counterfactual information may be particularly alluring because of its role in causal reasoning; its relationship with prediction and decision making; and its potential to fulfil emotion regulation and self-serving goals. Some future directions have been suggested, including investigating the role of individual differences in counterfactual curiosity on learning and wellbeing. This article is part of the theme issue ‘Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny’.

show abstract

Memory and decision making interact to shape the value of unchosen options

Cited by 29 publications

References 75 publications

The functional form of value normalization in human reinforcement learning

The functional form of value normalization in human reinforcement learning

How Working Memory and Reinforcement Learning Are Intertwined: A Cognitive, Neural, and Computational Perspective

Counterfactual curiosity: motivated thinking about what might have been

Contact Info

Product

Resources

About