Priors and Payoffs in Confidence Judgments

Abstract

<jats:label>1</jats:label><jats:title>Abstract</jats:title><jats:p>Priors and payoffs are known to affect perceptual decision-making, but little is understood about how they influence confidence judgments. For optimal perceptual decision-making, both priors and payoffs should be considered when selecting a response. However, for confidence to reflect the probability of being correct in a perceptual decision, priors should affect confidence but payoffs should not. To experimentally test whether human observers follow this normative behavior, we conducted an orientation-discrimination task with varied priors and payoffs, probing both perceptual and metacognitive decision-making. We then examined the placement of discrimination and confidence criteria according to several plausible Signal Detection Theory models. In the normative model, observers use the optimal discrimination criterion (i.e., the criterion that maximizes expected gain) and confidence criteria that shift with the discrimination criterion that maximizes accuracy (i.e., are not affected by payoffs). No observer was consistent with this model, with the majority exhibiting non-normative confidence behavior. One subset of observers ignored both priors and payoffs for confidence, always fixing the confidence criteria around the neutral discrimination criterion. The other group of observers incorrectly incorporated payoffs into their confidence by always shifting their confidence criteria with the same gains-maximizing criterion used for discrimination. Such metacognitive mistakes could have negative consequences outside the laboratory setting, particularly when priors or payoffs are not matched for all the possible decision alternatives.</jats:p>

Affective bias through the lens of Signal Detection Theory

“…Alternatively, information about priors and payoffs can be explicitly conveyed to the observer, but they still may choose to rely on the recent stimulus history for judging the context (Yu and Cohen, 2009). How observers come to form an estimate of their own perceptual sensitivity is less understood and rarely discussed (Rouault et al, 2019;Rouault and Fleming, 2020), but it has been proposed observers may make incorrect inferences about the underlying noise distributions when placing a criterion (Kubovy, 1977;Locke et al, 2020). For example, if the observer judges their d to be greater than it actually is (i.e., overconfidence) then this will lead to conservative criterion placement.…”

“…It is as if there is a cognitive cost to being incorrect and/or cognitive reward for being correct that is at play in perceptual decision-making. Another explanation for this phenomenon is that sacrificing some gains for better accuracy may foster more accurate judgements of perceptual confidence (Locke et al, 2020).…”

“…In the extreme, ignoring the reward values entirely would lead to the highest perceptual decision accuracy, but in practice observers are more likely to select a mixture strategy that is a trade-off between accuracy and gains Bohil, 1998, 2003). This trade-off behaviour in strategy selection can also be interpreted as a different need for accuracy: the need for our sense of confidence to best reflect true probability of being correct (Locke et al, 2020). The payoffs of a task can influence confidence ratings (Lebreton et al, 2018;Locke et al, 2020) despite not changing that probability that the decision was correct.…”

“…This trade-off behaviour in strategy selection can also be interpreted as a different need for accuracy: the need for our sense of confidence to best reflect true probability of being correct (Locke et al, 2020). The payoffs of a task can influence confidence ratings (Lebreton et al, 2018;Locke et al, 2020) despite not changing that probability that the decision was correct. For example, this can be an overconfidence when reporting the highly rewarded stimulus and underconfidence when reporting the low reward option.…”

“…Additionally, confidence judgements, on the correctness of their decision, could be included after every trial. Standard extensions of the SDT framework for confidence would then reveal if that observer let the rewards influence their confidence (Locke et al, 2020).…”