Neural Mechanisms for Adaptive Learned Avoidance of Mental Effort

Abstract

Humans tend to avoid mental effort. Previous studies have demonstrated this tendency using various demand-selection tasks; participants generally avoid options associated with higher cognitive demand. However, it remains unclear whether humans avoid mental effort adaptively in uncertain and non-stationary environments, and if so, what neural mechanisms underlie this learned avoidance and whether they remain the same irrespective of cognitive-demand types. We addressed these issues by developing novel demand-selection tasks where associations between choice options and cognitive-demand levels change over time, with two variations using mental arithmetic and spatial reasoning problems (29:4 and 18:2 males:females). Most participants showed avoidance, and their choices depended on the demand experienced on multiple preceding trials. We assumed that participants updated the expected cost of mental effort through experience, and fitted their choices by reinforcement learning models, comparing several possibilities. Model-based fMRI analyses revealed that activity in the dorsomedial and lateral frontal cortices was positively correlated with the trial-by-trial expected cost for the chosen option commonly across the different types of cognitive demand, and also revealed a trend of negative correlation in the ventromedial prefrontal cortex. We further identified correlates of cost-prediction-error at time of problem-presentation or answering the problem, the latter of which partially overlapped with or were proximal to the correlates of expected cost at time of choice-cue in the dorsomedial frontal cortex. These results suggest that humans adaptively learn to avoid mental effort, having neural mechanisms to represent expected cost and cost-prediction-error, and the same mechanisms operate for various types of cognitive demand.In daily life, humans encounter various cognitive demands, and tend to avoid high-demand options. However, it remains unclear whether humans avoid mental effort adaptively under dynamically changing environments, and if so, what are the underlying neural mechanisms and whether they operate irrespective of cognitive-demand types. To address these issues, we developed novel tasks, where participants could learn to avoid high-demand options under uncertain and non-stationary environments. Through model-based fMRI analyses, we found regions whose activity was correlated with the expected mental effort cost, or cost-prediction-error, regardless of demand-type, with overlap or adjacence in the dorsomedial frontal cortex. This finding contributes to clarifying the mechanisms for cognitive-demand avoidance, and provides empirical building blocks for the emerging computational theory of mental effort.

The Subjective Value of Cognitive Effort is Encoded by a Domain-General Valuation Network

“…Earlier studies have shown valuation regions encoding reward signals modulated by prior or anticipated cognitive demands (Botvinick et al, 2009;Satterthwaite et al, 2012;Schmidt et al, 2012;Schouppe et al, 2014a;Vassena et al, 2014;Dobryakova et al, 2017;Nagase et al, 2018). These results imply that the valuation network should also encode SV during cognitive effortbased decision-making.…”

“…This could reflect limited power to detect reward encoding: the dACC has elsewhere been shown to track both physical effort costs and reward amount (Harris and Lim, 2016;Klein-Flügge et al, 2016). Interestingly, however, numerous lines of evidence also suggest a somewhat more specialized role for these regions in processing cognitive effort costs including involvement in control modulation (Dosenbach et al, 2006), error awareness and processing (Klein et al, 2007), decisionmaking about physical effort (Croxson et al, 2009;Prévost et al, 2010;Kennerley et al, 2011;Kurniawan et al, 2013;Skvortsova et al, 2014), self-reported cognitive effort (in the AI; Otto et al, 2014), and learning about cognitive effort costs (Botvinick, 2007;Nagase et al, 2018).…”

The Subjective Value of Cognitive Effort is Encoded by a Domain-General Valuation Network

“…Earlier studies have shown that putative valuation regions reflect modulation of reward signals as a function of prior cognitive load, for example, or of incentive cues as a function of anticipated cognitive demands (Botvinick et al, 2009;Dobryakova et al, 2017;Nagase et al, 2018;Satterthwaite et al, 2012;Schmidt et al, 2012;Schouppe et al, 2014a;Vassena et al, 2014). These prior results imply that the domain-general valuation network should also encode SV during cognitive effort-based decision-making.…”

“…Also, unit recordings of monkey ACC neurons engaged in physical effort-based decision-making involves opposing signs in neighboring cells processing cost and benefit information (Kennerley et al, 2011) indicating that particular cost or benefit signals may cancel out at the resolution of fMRI. Interestingly, however, numerous lines of evidence also suggest a somewhat more specialized role for these regions in processing cognitive effort costs including their involvement in cognitive task attention and control modulation (Dosenbach et al, 2006), error awareness and processing (Klein et al, 2007), processing of affectively negative stimuli (Duerden et al, 2013), decision-making and learning about physical effort costs (Croxson et al, 2009;Kennerley et al, 2011;Kurniawan et al, 2013;Prévost et al, 2010;Skvortsova et al, 2014), self-reported cognitive effort ratings, in the case of the AI (Otto et al, 2014), and perhaps most pointedly, learning about cognitive effort costs (Botvinick, 2007;Nagase et al, 2018). Our results are thus consistent with the overarching hypothesis that the dACC and AI specialize in processing effort cost information in the service of SV computation.…”

Motivation and cognitive control in depression

“…In order to obtain trial-by-trial estimates of efficacy and reward rate, we fitted a reinforcement learning model (Gläscher, Daw, Dayan, & O'Doherty, 2010;Sutton & Barto, 1998) to the continuous (range: 0-1) subjective estimates of efficacy and reward rate (cf. Eldar, Hauser, Dayan, & Dolan, 2016;Nagase et al, 2018;Rutledge et al, 2014). The model assumed that the estimate of efficacy for the next trial (Et+1) depended on the current efficacy feedback (Et) and the prediction error (δt) weighted by a constant learning rate (α):…”