Conflict acts as an implicit cost in reinforcement learning

Cavanagh, James F.; Masters, Sean; Bath, Kevin G.; Frank, Michael J.

doi:10.1038/ncomms6394

Cited by 87 publications

(124 citation statements)

References 53 publications

Supporting

Mentioning

103

Contrasting

Order By: Relevance

“…To our knowledge, no one has investigated how the experience of conflict affects risky decision-making. These findings are in line with studies of the cost of effort, and they further support the idea that conflict acts as an implicit cost (Cavanagh et al, 2014). …”

Section: Resultssupporting

confidence: 89%

“…Cognitive conflict occurs when two competing response options compete for control of behavior (Botvinick, Braver, Barch, Carter, & Cohen, 2001), and it is reliably associated with dorsomedial activation (Shackman et al, 2011) and frontal midline theta (Cohen & Donner, 2013; Cohen, Ridderinkhof, Haupt, Elger, & Fell, 2008). Conflict can cause avoidance biases and stimulus devaluations (Cavanagh, Masters, Bath, & Frank, 2014; Dreisbach & Fischer, 2012; Fritz & Dreisbach, 2013; Schouppe, De Houwer, Richard Ridderinkhof, & Notebaert, 2012), and so can the similar phenomenon of stopping a pre-potent response (Wessel, O’Doherty, Berkebile, Linderman, & Aron, 2014; Wessel, Tonnesen, & Aron, 2015). Conflict can be reliably elicited and parametrically manipulated, suggesting that conflict can act as a succinct operationalization of dorsomedial computations affected by the implicit costs associated with effort.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Frontal theta accounts for individual differences in the cost of conflict on decision making

Pinner¹,

Cavanagh²

2017

Brain Research

Self Cite

View full text Add to dashboard Cite

Cognitive conflict is often experienced as a difficult, frustrating, and aversive state. Recent studies have indicated that conflict acts as an implicit cost during learning, valuation, and the instantiation of cognitive control. Here we investigated if an implicit manipulation of conflict also influences explicit decision making to risk. Participants were required to perform a Balloon Analogue Risk Task wherein the virtual balloon was inflated by performing a flankers task. By varying the percent of incongruent flanker trials between balloons, we hypothesized that participants would pump the balloon fewer times in conditions of higher conflict and that frontal midline theta would account for significant variance in this relationship. Across two studies, we demonstrate that conflict did not elicit reliable behavioral changes in this task across participants. However, individual differences in frontal theta power accounted for significant variance by predicting diminished balloon pumps. Thus, while conflict costs may act as investments to some individuals (invigorating behavior), it is aversive to others (diminishing behavior), and frontal midline theta power accounts for these varying behavioral tendencies between individuals. These findings demonstrate how frontal midline theta is not only a candidate mechanism for implementing cognitive control, but it is sensitive to the inherent costs therein.

show abstract

Section: Resultssupporting

confidence: 89%

Section: Introductionmentioning

confidence: 99%

Frontal theta accounts for individual differences in the cost of conflict on decision making

Pinner¹,

Cavanagh²

2017

Brain Research

Self Cite

View full text Add to dashboard Cite

show abstract

“…Similar baseline dependencies have been reported for receptor-specific noradrenergic drug effects [51,52] and for dopaminergic drug effects [79,80]. This baseline dependency may explain why previous studies did not find catecholaminergic drug effects on behavior at the group level [81,82].…”

Section: Discussionsupporting

confidence: 79%

Catecholaminergic Regulation of Learning Rate in a Dynamic Environment

et al. 2016

View full text Add to dashboard Cite

Adaptive behavior in a changing world requires flexibly adapting one’s rate of learning to the rate of environmental change. Recent studies have examined the computational mechanisms by which various environmental factors determine the impact of new outcomes on existing beliefs (i.e., the ‘learning rate’). However, the brain mechanisms, and in particular the neuromodulators, involved in this process are still largely unknown. The brain-wide neurophysiological effects of the catecholamines norepinephrine and dopamine on stimulus-evoked cortical responses suggest that the catecholamine systems are well positioned to regulate learning about environmental change, but more direct evidence for a role of this system is scant. Here, we report evidence from a study employing pharmacology, scalp electrophysiology and computational modeling (N = 32) that suggests an important role for catecholamines in learning rate regulation. We found that the P3 component of the EEG—an electrophysiological index of outcome-evoked phasic catecholamine release in the cortex—predicted learning rate, and formally mediated the effect of prediction-error magnitude on learning rate. P3 amplitude also mediated the effects of two computational variables—capturing the unexpectedness of an outcome and the uncertainty of a preexisting belief—on learning rate. Furthermore, a pharmacological manipulation of catecholamine activity affected learning rate following unanticipated task changes, in a way that depended on participants’ baseline learning rate. Our findings provide converging evidence for a causal role of the human catecholamine systems in learning-rate regulation as a function of environmental change.

show abstract

“…The most direct evidence comes from a recent pharmaco-genetic imaging study (Cavanagh et al, 2014). The paradigm was structured such that reward and punishment cues were accompanied by either high or low decision conflict (control demands), designed to test the prediction that conflict would attenuate reward and boost punishment learning.…”

Section: Da and Action Policy Learningmentioning

confidence: 99%

Dopamine Does Double Duty in Motivating Cognitive Effort

Westbrook

Braver

2016

Neuron

252

212

View full text Add to dashboard Cite

Cognitive control is subjectively costly, suggesting that engagement is modulated in relationship to incentive state. Dopamine appears to play key roles. In particular, dopamine may mediate cognitive effort by two broad classes of functions: 1) modulating the functional parameters of working memory circuits subserving effortful cognition, and 2) mediating value-learning and decision-making about effortful cognitive action. Here we tie together these two lines of research, proposing how dopamine serves “double duty”, translating incentive information into cognitive motivation.

show abstract

Conflict acts as an implicit cost in reinforcement learning

Cited by 87 publications

References 53 publications

Frontal theta accounts for individual differences in the cost of conflict on decision making

Frontal theta accounts for individual differences in the cost of conflict on decision making

Catecholaminergic Regulation of Learning Rate in a Dynamic Environment

Dopamine Does Double Duty in Motivating Cognitive Effort

Contact Info

Product

Resources

About