The impact of learning on perceptual decisions and its implication for speed-accuracy tradeoffs

Abstract

In standard models of perceptual decision-making, noisy sensory evidence is considered to be the primary source of choice errors and the accumulation of evidence needed to overcome this noise gives rise to speed-accuracy tradeoffs. Here, we investigated how the history of recent choices and their outcomes interacts with these processes using a combination of theory and experiment. We found that the speed and accuracy of performance of rats on olfactory decision tasks could be best explained by a Bayesian model that combines reinforcement-based learning with accumulation of uncertain sensory evidence. This model predicted the specific pattern of trial history effects that were found in the data. The results suggest that learning is a critical factor contributing to speed-accuracy tradeoffs in decision-making and that task history effects are not simply biases but rather the signatures of an optimal learning strategy.

Reinforcement biases subsequent perceptual decisions when confidence is low, a widespread behavioral phenomenon

“…Given that these trial-history effects diminish the overall reward return, and are hence suboptimal, the question is why they persist even after subjects are well trained in the task? Our results showed a similar phenomenon across various datasets and species that produced choice bias in perceptual decisions after rewarded decisions that were difficult and hence uncertain, consistent with recent reports (Mendonça et al, 2018;Lak et al, 2020). We show that these behavioral effects are normatively expected from various models that consider the uncertainty of stimulus states inherent in perceptual decisions.…”

Lapses in perceptual decisions reflect exploration

“…This means that animals should continue to use the outcomes of previous trials to update the values of different actions as long as this uncertainty persists. Such persistent learning has been observed in a number of studies (Busse et al, 2011; Lak et al, 2018; Mendonca et al, 2018; Odoemene et al, 2018; Pinto et al, 2018; Scott et al, 2015). The uncertainty-dependent exploration model predicts that changes in action values should manifest as changes in lapse rates.…”

“…Similarly, outcomes on a given trial can influence decisions about stimuli on subsequent trials through reinforcement learning, giving rise to serial biases. These biases occur even though the ideal observer should treat the evidence on successive trials as independent (Busse et al, 2011; Lak et al, 2018; Mendonca et al, 2018). When subjects can control how long they sample the stimulus, subjects maximizing reward rate may choose to make premature decisions, sacrificing accuracy for speed (Bogacz et al, 2006; Drugowitsch, DeAngelis, et al, 2014).…”

Optimal policy for multi-alternative decisions

“…For instance, the decision-maker must learn how to properly adjust the decision bounds to optimize the reward rate, which would result in trial-to-trial variability in the value of the bound. There is experimental evidence suggesting that learning can indeed induce extra variability in decision-making tasks 28 . Variability in bounds and neural responses could also be purposely induced by neural circuits to ensure that the decision-maker does not always choose the option with the highest value but also explores alternatives.…”