Individual differences in experienced and observational decision-making illuminate interactions between reinforcement learning and declarative memory

Abstract

Decision making can be shaped both by trial-and-error experiences and by memory of unique contextual information. Moreover, these types of information can be acquired either by means of active experience or by observing others behave in similar situations. The interactions between reinforcement learning parameters that inform decision updating and memory formation of declarative information in experienced and observational learning settings are, however, unknown. In the current study, participants took part in a probabilistic decision-making task involving situations that either yielded similar outcomes to those of an observed player or opposed them. By fitting alternative reinforcement learning models to each subject, we discerned participants who learned similarly from experience and observation from those who assigned different weights to learning signals from these two sources. Participants who assigned different weights to their own experience versus those of others displayed enhanced memory performance as well as subjective memory strength for episodes involving significant reward prospects. Conversely, memory performance of participants who did not prioritize their own experience over others did not seem to be influenced by reinforcement learning parameters. These findings demonstrate that interactions between implicit and explicit learning systems depend on the means by which individuals weigh relevant information conveyed via experience and observation.

Observational learning strategies impact the neural correlates of declarative memory formation

“…Previous work from our lab demonstrated that successfully discerning between one's own choices and the choices of others during reinforcement learning impacts the concurrent formation of declarative memories (Yifrah et al 2021). Building on these insights, we aimed to investigate the neural mechanisms that underlie observational vs. active reinforcement learning, and their connection with declarative memory formation.…”

“…The task, previously described in Yifrah et al (2021), comprised two interleaved trial types (Fig. 1A): experienced trials and observed trials.…”

“…In a previous behavioral study using a similar design as in the current study (Yifrah et al, 2021), we divided the participants into two groups, based on the difference between their learning modes, as re ected in their apparent learning behavior in experienced vs. observed trials, by applying a standard reinforcement learning model to the participants' choice sequence. There we found that participants who assigned greater weights to their own experience vs. those of others displayed enhanced memory performance for episodes involving signi cant reward prospects, compared to participants with similar learning rates for their own vs. others' choices, whose memory was not affected by reward prospects.…”

“…In a recent behavioral study, we found that participants whose learning process prioritizes their own experiences over choices made by others, displayed better memory performance and stronger subjective memory for events associated with high reward values. However, individuals who did not differentiate their own experiences from observed ones did not demonstrate an effect of reinforcement learning on memory performance (Yifrah et al 2021). These ndings suggest that interactions between implicit and explicit learning systems are in uenced by the manner in which individuals are able to distinguish among different sources of information.…”

How Working Memory and Reinforcement Learning Are Intertwined: A Cognitive, Neural, and Computational Perspective

“…Counterfactual learning of items chosen against one another is modulated by the strength of the episodic memory for them (Biderman & Shohamy, 2021). In some people, memory strength and RL learning rate seem to trade off depending on experimental learning context (Yifrah et al, 2021). This relationship between long term memory and RL isn't particularly surprising, considering the importance a long-term storage would have on more realistic environments, which have highdimensional, continuous, and partially-observable state spaces.…”