Transfer of learned cognitive flexibility to novel stimuli and task sets

Abstract

Adaptive behavior requires learning about the structure of the environment to derive optimal action policies, and previous studies have documented transfer of such structural knowledge to bias choices in new environments. Here, we asked whether people could also acquire and transfer more abstract knowledge across different task environments, in particular, expectations about demands on cognitive control. Over three experiments, participants performed a probabilistic card-sorting task in environments of either a low or high volatility of task rule changes (requiring low or high cognitive flexibility) before transitioning to a medium-volatility environment. Using reinforcement learning modeling, we consistently found that previous exposure to high task rule volatility led to faster adaptation to rule changes in the subsequent transfer phase. This transfer of expectations about demands on cognitive flexibility was both task- (Experiment 2) and stimulus- (Experiment 3) independent, thus demonstrating the formation and generalization of environmental structure knowledge to guide cognitive control.

Learning environment-specific learning rates

“…This is consistent with several previous studies showing that people seem to weigh unexpected positive versus negative outcomes separately when updating action values (Gershman, 2015;Niv et al, 2012;& Lebreton, 2022;Rosenbaum et al, 2022). Perhaps most noteworthy, this is also in line with another recent study that investigated inertia of used learning rates into a separate subsequent transfer phase (Wen et al, 2023). Specifically, they manipulated volatilities between subjects (i.e., each subject was given one volatility level) in a Wisconsin Card Sorting Task (WCST).…”

“…Most importantly, we compared models with either a shared learning rate across both environments, versus models with different, environment-specific learning rates for both casinos. Furthermore, we compared models with separate learning rates after positive and negative reward feedback (Gershman, 2015;Niv et al, 2012;Palminteri & Lebreton, 2022;Rosenbaum et al, 2022, Wen et al, 2023, to models with one learning rate for both feedback types. To quantify the alternative hypothesis that learning rate changes merely reflect local adaptations to the experienced prediction errors, we also implemented models with variable learning rates that depend on recently experienced prediction errors (Bai et al, 2014;Krugel et al, 2009;Li et al, 2011;Pearce & Hall, 1980).…”

“…According to this model, the posterior probabilities that positive and negative learning rates were higher in the formerly volatile than in the formerly stable environment was 0.263 and 0.95, respectively (Figure 6A). Secondly, we also tested whether the proportion of participants who chose the correct slot machine (i.e., the slot machine with the highest likelihood of yielding reward), was significantly different in both environments on the first trials following the first reward contingency switch in the test phase (see also Wen et al, 2023). Since this analysis is explorative, we did not correct for multiple comparisons.…”

Contextual overtraining accelerates habit formation in new stimuli

“…While a stable context is considered a necessary condition for habits to form and be expressed [3][4][5] , the influence of the learned higher-order properties of the task context on habit learning and expression have not yet been studied. Computational and behavioural work on cognitive flexibility shows that humans engage in such meta-learning, and use this knowledge to optimize performance [6][7][8][9][10] . For example, environments where S-R-O contingencies frequently change encourage higher learning rates than more stable contexts 6 .…”