Flexible use of confidence to guide advice requests

Shenhav

2020

Preprint

To determine how much cognitive control to invest in a task, people need to consider whether exerting control matters for obtaining potential rewards. In particular, they need to account for the efficacy of their performance – the degree to which potential rewards are determined by their performance or by independent factors (e.g., random chance). Yet it remains unclear how people learn about their performance efficacy in a given environment. Here, we examined the neural and computational mechanisms through which people (a) learn and dynamically update efficacy expectations in a changing environment, and (b) proactively adjust control allocation based on their current efficacy expectations. We recorded EEG in 40 participants performing an incentivized cognitive control task, while their performance efficacy (the likelihood that reward for a given trial would be determined by performance or at random) dynamically varied over time. We found that participants continuously updated their self- reported efficacy expectations based on recent feedback, and that these updates were well described by a standard prediction error-based reinforcement learning algorithm. Paralleling findings on updating of expected rewards, we found that model-based estimates of efficacy prediction errors were encoded by the feedback-related P3b. Updated expectations of efficacy in turn influenced levels of effort exerted on subsequent trials, reflected in greater proactive control (indexed by the contingent negative variation [CNV]) and improved performance when participants expected their performance to be more efficacious. These findings demonstrate that learning and adaptation to the efficacy of one’s environment is underpinned by similar computations and neural mechanisms as are involved in learning about potential reward.

Section: Discussionmentioning

confidence: 99%

Learning when effort matters: Neural dynamics underlying updating and adaptation to changes in performance efficacy

Grahek

Shenhav

2020

Preprint

“…Similar to its role in regulating learning of transition probabilities ( Meyniel et al, 2015 ; Meyniel and Dehaene, 2017 ), information seeking/exploration in decision making ( Desender et al, 2018a ; Boldt et al, 2019 ), and hierarchical reasoning ( Sarafyazd and Jazayeri, 2019 ), people could leverage confidence to calibrate their use of online predictions. In line with this suggestion, people learn more about advice givers when they are more confident in the choices that advice is about ( Carlebach and Yeung, 2020 ). In the throwing example above, the more confident you are about the exact landing position of the dart, the more surprised you should be when you find that landing position to be different: The more confident you are, the more evidence you have that your internal model linking angles to landing positions is wrong, and the more information you get about how this model is wrong.…”

Section: Introductionmentioning

confidence: 91%

Response-based outcome predictions and confidence regulate feedback processing and learning

Nassar

Bruckner

et al. 2021

eLife

Self Cite

Influential theories emphasize the importance of predictions in learning: we learn from feedback to the extent that it is surprising, and thus conveys new information. Here, we explore the hypothesis that surprise depends not only on comparing current events to past experience, but also on online evaluation of performance via internal monitoring. Specifically, we propose that people leverage insights from response-based performance monitoring – outcome predictions and confidence – to control learning from feedback. In line with predictions from a Bayesian inference model, we find that people who are better at calibrating their confidence to the precision of their outcome predictions learn more quickly. Further in line with our proposal, EEG signatures of feedback processing are sensitive to the accuracy of, and confidence in, post-response outcome predictions. Taken together, our results suggest that online predictions and confidence serve to calibrate neural error signals to improve the efficiency of learning.

“…Just as rewards can serve as training signals to guide future choices, so can confidence and conflict (Yeung & Summerfield, 2012). For instance, people use confidence as a proxy for feedback when actual feedback is not available or delayed, as is often the case in real world decisions (Carlebach & Yeung, 2020a;Guggenmos, Wilbertz, Hebart, & Sterzer, 2016;Ptasczynski, Steinecker, Sterzer, & Guggenmos, 2021). High confidence -or ease -then can serve as a positive reinforcer (Winkielman, Schwarz, Fazendeiro, & Reber, 2003).…”

Section: D)mentioning

confidence: 99%

Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making

Neuroscience & Biobehavioral Reviews

Shenhav

2022

While often seeming to investigate rather different problems, research into value-based decision making and cognitive control have historically offered parallel insights into how people select thoughts and actions. While the former studies how people weigh costs and benefits to make a decision, the latter studies how they adjust information processing to achieve their goals.Recent work has highlighted ways in which decision-making research can inform our understanding of cognitive control. Here, we provide the complementary perspective: how cognitive control research has informed understanding of decision-making. We highlight three particular areas of research where this critical interchange has occurred: (1) how different types of goals shape the evaluation of choice options, (2) how people use control to adjust how they make their decisions, and (3) how people monitor decisions to inform adjustments to control at multiple levels and timescales. We show how adopting this alternate viewpoint offers new insight into the determinants of both decisions and control; provides alternative interpretations for common neuroeconomic findings; and generates fruitful directions for future research.