Reinforcement learning of adaptive control strategies

Abstract

Humans can up- or downregulate the degree to which they rely on task information for goal directed behaviour, a process often referred to as cognitive control. Adjustments in cognitive control are traditionally studied in response to experienced or expected task-rule conflict. However, recent theories suggest that people can also learn to adapt control settings through reinforcement. Across three preregistered task switching experiments (n=415), we selectively rewarded correct performance on trials with either more (incongruent) or less (congruent) task-rule conflict. Results confirmed the hypothesis that people rewarded more on incongruent trials showed smaller task-rule congruency effects, thus optimally adapting their control settings to the reward scheme. Using drift diffusion modelling, we further show that this reinforcement of cognitive control may occur through conflict-dependent within-trial adjustments of response thresholds after conflict detection. Together, our findings suggest that, while people remain more efficient at learning stimulus-response associations through rewards, they can similarly learn cognitive control strategies through reinforcement.

The Temporal Dynamics of Metacognitive Experiences Track Rational Adaptations in Task Performance

“…Moreover, there are reasons to assume that it is more plausible that people adapt their decision boundary rather than their peak amplitude. The decision boundary is typically considered a strategic parameter that can be manipulated via instructions 81 and in response to conflict 33,[82][83][84][85][86][87] . On the other hand, behavioral measures for irrelevant capture (interference effects, which are here quantified by peak amplitude) are often utilized to measure individual differences in inhibitory capacity or executive functioning, implying a closer association with one's ability rather than strategic adjustments 74,[88][89][90][91] .…”

The effects of reinforcing task alternations on voluntary task selection

“…Stimuli consisted of 400 unique Dutch words that can be categorized both according to their size and animacy, presented in white (30px Verdana) on a black background. They were adapted from Schneider (2015) and Braem (2017), and extended by 80 novel items, selected from a Dutch database (Keuleers et al, 2010) to match them in frequency and length per word category (e.g., small and animate, large and animate (see also Held et al, 2023). We included the same number of words per category in the experiment and each of the five experimental blocks consisted of a similar amount of task switches and task repetitions.…”

Expectation modulates learning emotional words: Evidence from a hierarchical Bayesian model

“…These expectations subsequently guide their successive behaviours. Previous researchers have combined reinforcement learning with studies in other fields (Li et al, 2023;Liu et al, 2023;Hackel et al, 2020;Held et al, 2024;Lockwood & Klein-Flügge, 2021;Wang et al, 2023;Zhang et al, 2023). For instance, Wang et al applied reinforcement learning to emotion processing and elaborated on the cognitive neuroscientific mechanisms of how fear interferes with adaptation to environmental volatility in dynamic environments at the computational level.…”