Adaptive History Biases Result from Confidence-weighted Accumulation of Past Choices

Abstract

Perceptual decision-making is biased by previous events, including the history of preceding choices: Observers tend to repeat (or alternate) their judgments of the sensory environment more often than expected by chance. Computational models postulate that these so-called choice history biases result from the accumulation of internal decision signals across trials. Here, we provide psychophysical evidence for such a mechanism and its adaptive utility. Male and female human observers performed different variants of a challenging visual motion discrimination task near psychophysical threshold. In a first experiment, we decoupled categorical perceptual choices and motor responses on a trial-by-trial basis. Choice history bias was explained by previous perceptual choices, not motor responses, highlighting the importance of internal decision signals in action-independent formats. In a second experiment, observers performed the task in stimulus environments containing different levels of autocorrelation and providing no external feedback about choice correctness. Despite performing under overall high levels of uncertainty, observers adjusted both the strength and the sign of their choice history biases to these environments. When stimulus sequences were dominated by either repetitions or alternations, the individual degree of this adjustment of history bias was about as good a predictor of individual performance as individual perceptual sensitivity. The history bias adjustment scaled with two proxies for observers’ confidence about their previous choices (accuracy and reaction time). Taken together, our results are consistent with the idea that action-independent, confidence-modulated decision variables are accumulated across choices in a flexible manner that depends on decision-makers’ model of their environment.Significance statementDecisions based on sensory input are often influenced by the history of one’s preceding choices, manifesting as a bias to systematically repeat (or alternate) choices. We here provide support for the idea that such choice history biases arise from the context-dependent accumulation of a quantity referred to as the decision variable: the variable’s sign dictates the choice and its magnitude the confidence about choice correctness. We show that choices are accumulated in an actionindependent format and a context-dependent manner, weighted by the confidence about their correctness. This confidence-weighted accumulation of choices enables decision-makers to flexibly adjust their behavior to different sensory environments. The bias adjustment can be as important for optimizing performance as one’s sensitivity to the momentary sensory input.

Reinforcement biases subsequent perceptual decisions when confidence is low, a widespread behavioral phenomenon

“…We show that these behavioral effects are normatively expected from various models that consider the uncertainty of stimulus states inherent in perceptual decisions. It is worth noting that the confidence-gauged learning described here requires observing the trial feedback, and it might thus differ from sequential choice effects in the absence of trial feedback both experimentally (Braun et al, 2018), and from a computational perspective (Glaze et al, 2015). Moreover, confidence-dependent learning differs from history-effects for highly discriminable stimuli, i.e.…”

“…There is mounting evidence that in perceptual decision making tasks, even though reward is only contingent on accurate judgment about the current sensory stimulus, choices can be influenced by previous trials across species (Abrahamyan et al, 2016;Akaishi et al, 2014;Akrami et al, 2018;Braun et al, 2018;Busse et al, 2011;Cho et al, 2002;Fan et al, 2018;Fischer and Whitney, 2014;Fritsche et al, 2017;Frund et al, 2014;Gold et al, 2008;Hwang et al, 2017;Lueckmann et al, 2018;Luu and Stocker, 2018;Marcos et al, 2013;Tsunada et al, 2019;Urai et al, 2017). Several such studies have shown that subjects might repeat the previously rewarded choice or avoid it after an unsuccessful trial, suggesting that basic forms of reward-based learning are at work even at asymptotic, steady-state perceptual performance.…”

Evidence integration and decision-confidence are modulated by stimulus consistency

“…Previous research has reported sequential effects operating at the trial level. For example, a choice biases the interpretation of evidence in subsequent trials (Abrahamyan et al, 2016;Braun et al, 2018;Urai et al, 2019;Yu & Cohen, 2009). Similarly, a preliminary decision biases processing of additional post-choice evidence towards confirming the initial decision (Bronfman et al, 2015;Talluri et al, 2018;Rollwage et al, 2020), and decisions bias the strength-evaluation of pro-choice evidence that led to it (Jazayeri & Movshon, 2007;Luu & Stocker, 2018;Stocker & Simoncelli, 2008).…”

A normative account of choice history effects in mice and humans

“…Indeed, in contrast to perceptual decision-making tasks that expose animals to a static environment, models that assume a dynamic environment are better at capturing the animals' choices (Mendonca et al, 2020). However, when an environment is not static and task-relevant stimuli exhibit autocorrelations, choice history effects carry an important adaptive function (Braun, Urai, & Donner, 2018). The experimental data…”