The impact of learning on perceptual decisions and its implication for speed-accuracy tradeoffs

Mendonça, André G.; Drugowitsch, Jan; Vicente, M. Inês; DeWitt, Eric; Pouget, Alexandre; Mainen, Zachary F.

doi:10.1038/s41467-020-16196-7

Cited by 39 publications

(43 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…It would be interesting to apply a modified version of the DT model to perceptual decision-making tasks to attempt to capture such lapse behavior. Furthermore, in perceptual decision-making tasks that expose animals to a static environment, models that assume a dynamic environment are better at capturing the animals’ choices [ 48 ]. However, when an environment is not static and task-relevant stimuli exhibit autocorrelations or bias, choice history effects carry an important adaptive function [ 49 – 51 ].…”

Section: Discussionmentioning

confidence: 99%

Choice history effects in mice and humans improve reward harvesting efficiency

et al. 2021

View full text Add to dashboard Cite

Choice history effects describe how future choices depend on the history of past choices. In experimental tasks this is typically framed as a bias because it often diminishes the experienced reward rates. However, in natural habitats, choices made in the past constrain choices that can be made in the future. For foraging animals, the probability of earning a reward in a given patch depends on the degree to which the animals have exploited the patch in the past. One problem with many experimental tasks that show choice history effects is that such tasks artificially decouple choice history from its consequences on reward availability over time. To circumvent this, we use a variable interval (VI) reward schedule that reinstates a more natural contingency between past choices and future reward availability. By examining the behavior of optimal agents in the VI task we discover that choice history effects observed in animals serve to maximize reward harvesting efficiency. We further distil the function of choice history effects by manipulating first- and second-order statistics of the environment. We find that choice history effects primarily reflect the growth rate of the reward probability of the unchosen option, whereas reward history effects primarily reflect environmental volatility. Based on observed choice history effects in animals, we develop a reinforcement learning model that explicitly incorporates choice history over multiple time scales into the decision process, and we assess its predictive adequacy in accounting for the associated behavior. We show that this new variant, known as the double trace model, has a higher performance in predicting choice data, and shows near optimal reward harvesting efficiency in simulated environments. These results suggests that choice history effects may be adaptive for natural contingencies between consumption and reward availability. This concept lends credence to a normative account of choice history effects that extends beyond its description as a bias.

show abstract

Section: Discussionmentioning

confidence: 99%

Choice history effects in mice and humans improve reward harvesting efficiency

et al. 2021

View full text Add to dashboard Cite

show abstract

“…That discrete states underpin mouse choice behavior may also call for new normative models to explain why mice may develop these states to begin with [1]. The existence of disengaged and engaged states could reflect explore-exploit behavior [15, 16, 50], optimal learning (e.g., [4, 23, 26, 47]), or could simply indicate incomplete learning of the task. Another promising direction will be to replace the model’s fixed transition matrix with a generalized linear model allowing external covariates to modulate the probability of state changes [12]).…”

Section: Discussionmentioning

confidence: 99%

Mice alternate between discrete strategies during perceptual decision-making

Ashwood

Roy

Stone

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Classical models of perceptual decision-making assume that animals use a single, consistent strategy to integrate sensory evidence and form decisions during an experiment. Here we provide analyses showing that this common view is incorrect. We use a latent variable modeling framework to show that decision-making behavior in mice reflects an interplay between different strategies that alternate on a timescale of tens to hundreds of trials. This model provides a powerful alternate explanation for "lapses" commonly observed during psychophysical experiments. Formally, our approach consists of a Hidden Markov Model (HMM) with states corresponding to different decision-making strategies, each parameterized by a distinct Bernoulli generalized linear model (GLM). We fit the resulting model (GLM-HMM) to choice data from two large cohorts of mice in different perceptual decision-making tasks. For both datasets, we found that mouse decision-making was far better described by a GLM-HMM with 3 or 4 states than by a traditional psychophysical model with lapses. The identified states were highly consistent across animals, consisting of a single "engaged" state, in which the strategy relied heavily on the sensory stimulus, and multiple biased or disengaged states in which accuracy was low. These states persisted for many trials, suggesting that lapses were not independent, but reflected state dynamics in which animals were relatively engaged or disengaged for extended periods of time. We found that for most animals, response times and violation rates were positively correlated with disengagement, providing independent correlates of the identified changes in strategy. The GLM-HMM framework thus provides a powerful lens for the analysis of decision-making, and suggests that standard measures of psychophysical performance mask the presence of slow but dramatic alternations in strategy across trials.

show abstract

“…The MF correction exploits the idea that the effect of slow drifts should be similar for at least a small stretch of trials (n>=2) and any influence of future rewards on past choices is acausal, likely due to slow drifts and needs to be removed. (Mendonça et al 2020) proposed the variant of the correction for choice-outcome biases and (Lak, Hueske, et al 2020) for stimulus history biases. We describe and test the efficacy of both these variants here.…”

Section: Correction In Actionmentioning

confidence: 99%

“…This dependence is thought to arise from systematic updating of decision variables from trial to trial. These updates may reflect ongoing learning (Dayan and Daw 2008), for instance an agent learning to perform a perceptual categorization task might update its beliefs about the prior probabilities of the different categories (Yu and Cohen 2008;Zhang, Huang, and Yu 2014), the category boundary separating them (Drugowitsch et al 2019;Mendonça et al 2020), or the values of the available actions (Lak, Okun, et al 2020;Lak, Hueske, et al 2020;Pisupati et al 2021). Alternatively, systematic updates may reflect heuristic strategies adopted by decision-makers due to resource constraints or mismatched objectives (Gigerenzer and Gaissmaier 2011;Abrahamyan et al 2016;Gardner 2019).…”

Section: Introductionmentioning

confidence: 99%

“…The authors showed that a slow drift in criterion produces an apparent history dependence that mimics the signatures of updates from a systematic learning process. Hence, in order to remove the influence of these slow drifts, they proposed a model-free (MF) correction (Lak, Hueske, et al 2020), similar in spirit to an approach employed by (Mendonça et al 2020). They reasoned that slow drifts would produce correlations in choices across multiple successive trials.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Limitations of a proposed correction for slow drifts in decision criterion

Gupta

Brody

2021

Preprint

View full text Add to dashboard Cite

Trial history biases in decision-making tasks are thought to reflect systematic updates of decision variables, therefore their precise nature informs conclusions about underlying heuristic strategies and learning processes. However, random drifts in decision variables can corrupt this inference by mimicking the signatures of systematic updates. Hence, identifying the trial-by-trial evolution of decision variables requires methods that can robustly account for such drifts. Recent studies (Lak 20, Mendonça 20) have made important advances in this direction, by proposing a convenient method to correct for the influence of slow drifts in decision criterion, a key decision variable. Here we apply this correction to a variety of updating scenarios, and evaluate its performance. We show that the correction fails for a wide range of commonly assumed systematic updating strategies, distorting one's inference away from the veridical strategies towards a narrow subset. To address these limitations, we propose a model-based approach for disambiguating systematic updates from random drifts, and demonstrate its success on real and synthetic datasets. We show that this approach accurately recovers the latent trajectory of drifts in decision criterion as well as the generative systematic updates from simulated data. Our results offer recommendations for methods to account for the interactions between history biases and slow drifts, and highlight the advantages of incorporating assumptions about the generative process directly into models of decision-making.

show abstract

The impact of learning on perceptual decisions and its implication for speed-accuracy tradeoffs

Cited by 39 publications

References 53 publications

Choice history effects in mice and humans improve reward harvesting efficiency

Choice history effects in mice and humans improve reward harvesting efficiency

Mice alternate between discrete strategies during perceptual decision-making

Limitations of a proposed correction for slow drifts in decision criterion

Contact Info

Product

Resources

About