Neural systems underlying the learning of cognitive effort costs

Abstract

People balance the benefits of cognitive work against the costs of cognitive effort. Models that incorporate prospective estimates of the costs of cognitive effort into decision making require a mechanism by which these costs are learned. However, it remains an open question what brain systems are important for this learning, particularly when learning is not tied explicitly to a decision about what task to perform. In this fMRI experiment, we parametrically manipulated the level of effort a task requires by increasing task switching frequency across six task contexts. In a scanned learning phase, participants implicitly learned about the task switching frequency in each context. In a subsequent test phase, participants made selections between pairs of these task contexts. We modeled learning within a reinforcement learning framework, and found that effort expectations that derived from task-switching probability and response time (RT) during learning were the best predictors of later choice behavior. Prediction errors (PE) from these two models were associated with FPN during distinct learning epochs. Specifically, PE derived from expected RT was most correlated with the fronto-parietal network early in learning, whereas PE derived from expected task switching frequency was correlated with the fronto-parietal network late in learning. These results suggest that multiple taskrelated factors are tracked by the brain while performing a task that can drive subsequent estimates of effort costs.

The Controllosphere: The Neural Origin of Cognitive Effort

“…Further, extensive neuroimaging and neurophysiological evidence indicates that ACC is especially sensitive to cognitive effort and fatigue (Engstrom et al, 2015;Massar et al, 2015;Muller & App, 2019;Persson et al, 2013;Shenhev et al, 2017;Vassena et al, 2017). For example, application of transcranial THE CONTROLLOSPHERE brain stimulation over human dorsomedial prefrontal cortex (including ACC) modulates the willingness of subjects to apply mental effort (Soutschek et al, 2022), and ACC prediction errors are associated with learning the costs of cognitive effort (Sayali & Badre, 2021). Notably, an RNN model of ACC predicts upcoming cognitive states based on information about the current state, eliciting prediction errors to unexpected changes in state .…”

Expected Costs of Mental Efforts are Updated When People Exert Effort, not by Prospective Information

“…Further, TD learning was reported as the learning algorithm of pain and loss of reward, just as for rewards, although the PE of these positive punishments is thought to be encoded with opposite signs to reward PE signals in the human striatum (Seymour et al ., 2007; Seymour et al ., 2004; Sutton and Barto, 2018). Modeling of behavioural and neural data indicate that mental effort costs respect RL algorithms (Nagase et al ., 2018; Sayali and Badre, 2019; Sayali and Badre, 2021). Collectively, these prior studies imply that human brains learn mental effort cost as an integrated value in the manner of TD learning.…”

“…Striatal dopamine dynamics reflect a dissociation between the learning of reward values and physical effort costs (Gan et al, 2010). Although studies have consistently shown that humans avoid mental effort, few have investigated the neural mechanisms by which mental effort costs are learned (Cavanagh et al, 2014; Kool et al, 2010; Nagase et al, 2018; Sayali and Badre, 2019; Sayali and Badre, 2021; Westbrook et al, 2013). Therefore, little is known about what mechanism underlies adaptive learning to avoid mental effort.…”

“…Yet, these models do not make explicit assumptions about the dynamics by which human brains learn mental effort costs. There is evidence that brains signal a cost prediction error (CPE), estimated by reinforcement learning (RL) algorithms, of mental effort demand (Nagase et al, 2018; Sayali and Badre, 2019; Sayali and Badre, 2021). Previous evidence showed that the learning of negatively-valanced stimuli such as pain, air puff, or reward omission follows temporal-difference (TD) learning (Matsumoto and Hikosaka, 2009; Schultz et al, 1997; Seymour et al, 2004).…”

“…Yet, these models do not make explicit assumptions about the dynamics by which human brains learn mental effort costs. There is evidence that brains signal a cost prediction error (CPE), estimated by reinforcement learning (RL) algorithms, of mental effort demand (Nagase et al, 2018;Sayali and Badre, 2019;Sayalı and Badre, 2021).…”