Prefrontal cortex represents heuristics that shape choice bias and its integration into future behavior

Mochol, Gabriela; Kiani, Roozbeh; Moreno‐Bote, Rubén

doi:10.1016/j.cub.2021.01.068

Cited by 47 publications

(56 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If choice history effects emerge as an evolutionary adaptation to natural habitats, then the full suppression of this behavior, even in well-trained animals, in a perceptual or value-based decision-making tasks might be difficult. Indeed, in many decision-making tasks that do not impose contingencies of previous choices on current choices, choice history effects persist [ 1 , 3 , 6 , 22 , 46 ]. Understanding the behavioral mechanisms that generate choice-history effects in these tasks may require the manipulation of behavioral contingencies, linking past choices to current choices in a parametric way.…”

Section: Discussionmentioning

confidence: 99%

“…We are not aware of any published work that describes the single neuron correlates of the choice history for past two to 15 trials captured by the slow component of choice history effects. However, slow and fast choice history single neuron representations, albeit at the different time scales (slow history effect comprised trials past within the 130–400 range), have been described in monkey prefrontal areas [ 46 ]. This suggests that the same neural circuits can capture both time scale processes in VI task.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Choice history effects in mice and humans improve reward harvesting efficiency

et al. 2021

View full text Add to dashboard Cite

Choice history effects describe how future choices depend on the history of past choices. In experimental tasks this is typically framed as a bias because it often diminishes the experienced reward rates. However, in natural habitats, choices made in the past constrain choices that can be made in the future. For foraging animals, the probability of earning a reward in a given patch depends on the degree to which the animals have exploited the patch in the past. One problem with many experimental tasks that show choice history effects is that such tasks artificially decouple choice history from its consequences on reward availability over time. To circumvent this, we use a variable interval (VI) reward schedule that reinstates a more natural contingency between past choices and future reward availability. By examining the behavior of optimal agents in the VI task we discover that choice history effects observed in animals serve to maximize reward harvesting efficiency. We further distil the function of choice history effects by manipulating first- and second-order statistics of the environment. We find that choice history effects primarily reflect the growth rate of the reward probability of the unchosen option, whereas reward history effects primarily reflect environmental volatility. Based on observed choice history effects in animals, we develop a reinforcement learning model that explicitly incorporates choice history over multiple time scales into the decision process, and we assess its predictive adequacy in accounting for the associated behavior. We show that this new variant, known as the double trace model, has a higher performance in predicting choice data, and shows near optimal reward harvesting efficiency in simulated environments. These results suggests that choice history effects may be adaptive for natural contingencies between consumption and reward availability. This concept lends credence to a normative account of choice history effects that extends beyond its description as a bias.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Choice history effects in mice and humans improve reward harvesting efficiency

et al. 2021

View full text Add to dashboard Cite

show abstract

“…We next wondered whether the model exhibits history effects. Recent work has shown sensory history-dependent biases in decision-making on both short (a few trials) and long time scales (∼100 trials) (Mochol et al 2021;Tervo et al 2021). We expect history effects in the model, because the plastic weights adapt to newly observed samples, redistributing the probability mass continually (Fig S3).…”

Section: The Model Exhibits Long-and Short-term History Effectsmentioning

confidence: 93%

Long- and short-term history effects in a spiking network model of statistical learning

Maes

Barahona

Clopath

2021

Preprint

View full text Add to dashboard Cite

The statistical structure of the environment is often important when making decisions. There are multiple theories of how the brain represents statistical structure. One such theory states that neural activity spontaneously samples from probability distributions. In other words, the network spends more time in states which encode high-probability stimuli. Existing spiking network models implementing sampling lack the ability to learn the statistical structure from observed stimuli and instead often hard-code a dynamics. Here, we focus on how arbitrary prior knowledge about the external world can both be learned and spontaneously recollected. We present a model based upon learning the inverse of the cumulative distribution function. Learning is entirely unsupervised using biophysical neurons and biologically plausible learning rules. We show how this prior knowledge can then be accessed to compute expectations and signal surprise in downstream networks. Sensory history effects emerge from the model as a consequence of ongoing learning.

show abstract

“…Neurons in the central-lateral primate OFC and rat lOFC have been shown to represent the values associated with these sensory cues in their firing rates, as well as reward outcomes and outcome values (Thorpe et al, 1983;Schoenbaum et al, 1998;Tremblay and Schultz, 1999 In this study, we have described dynamic trial-by-trial changes in firing rates that reflected reward history just preceding the choice. These reward history representations might influence ongoing neural dynamics supporting the upcoming choice (Mochol et al, 2021), and/or mediate learning. In contrast to broadly distributed adaptive value coding, this activity was restricted to a particular subset of neurons that were identifiable by two independent clustering methods, and that exhibited the strongest encoding of reward outcomes.…”

Section: Representations Of Reward History In Ofcmentioning

confidence: 99%

Subpopulations of neurons in lOFC encode previous and current rewards at time of choice

Hocker

Brody

Savin

et al. 2021

eLife

View full text Add to dashboard Cite

Studies of neural dynamics in lateral orbitofrontal cortex (lOFC) have shown that subsets of neurons that encode distinct aspects of behavior, such as value,may project to common downstreamtargets. However, it is unclear whether reward history, which may subserve lOFC's well-documented role in learning, is represented by functional subpopulations in lOFC. Previously, we analyzed neural recordings from rats performing a value-based decision-making task, and we documented trial-by-trial learning that required lOFC (Constantinople et al., 2019). Here we characterize functional subpopulations of lOFC neurons during behavior, including their encoding of task variables. We found five distinct clusters of lOFC neurons, either based on clustering of their trial-averaged peristimulus time histograms (PSTHs), or a feature space defined by their average conditional firing rates aligned to different task variables. We observed weak encoding of reward attributes, but stronger encoding of reward history, the animal's left or right choice, and reward receipt across all clusters. Only one cluster, however, encoded the animal's reward history at the time shortly preceding the choice, suggesting a possible role in integrating previous and current trial outcomes at the time of choice. This cluster also exhibits qualitatively similar responses to identified corticostriatal projection neurons in a recent study (Hirokawa et al., 2019), and suggests a possible role for subpopulations of lOFC neurons in mediating trial-by-trial learning.

show abstract

Prefrontal cortex represents heuristics that shape choice bias and its integration into future behavior

Cited by 47 publications

References 56 publications

Choice history effects in mice and humans improve reward harvesting efficiency

Choice history effects in mice and humans improve reward harvesting efficiency

Long- and short-term history effects in a spiking network model of statistical learning

Subpopulations of neurons in lOFC encode previous and current rewards at time of choice

Contact Info

Product

Resources

About