A repertoire of foraging decision variables in the mouse brain

Cazettes, Fanny; Murakami, Masayoshi; Renart, Alfonso; Mainen, Zachary F.

doi:10.1101/2021.04.01.438090

Cited by 13 publications

(27 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Whether the brain of the rat uses the lack of reward as a cue to disregard the information about the transition bias, yet maintaining it in a private subspace, or whether it directly eliminates such information remains to be tested. On the one hand, areas of the brain such as the prefrontal cortex seem to keep task-irrelevant information while performing a given task, presumably to have access to it, in case it becomes useful in the future (Barraclough, Conroy, and Lee 2004; Xiong, Znamenskiy, and Zador 2015; Cazettes et al 2021). On the other hand, representing information in the spiking activity of populations of neurons is costly.…”

Section: Discussionmentioning

confidence: 99%

Ecologically pre-trained RNNs explain suboptimal animal decisions

Molano-Mazón

Duque

et al. 2021

Preprint

View full text Add to dashboard Cite

When faced with a new task, animals′ cognitive capabilities are determined both by individual experience and by structural priors evolved to leverage the statistics of natural environments. Rats can quickly learn to capitalize on the trial sequence correlations of two-alternative forced choice (2AFC) tasks after correct trials, but consistently deviate from optimal behavior after error trials, when they waive the accumulated evidence. To understand this outcome-dependent gating, we first show that Recurrent Neural Networks (RNNs) trained in the same 2AFC task outperform animals as they can readily learn to use previous trials′ information both after correct and error trials. We hypothesize that, while RNNs can optimize their behavior in the 2AFC task without a priori restrictions, rats′ strategy is constrained by a structural prior adapted to a natural environment in which rewarded and non-rewarded actions provide largely asymmetric information. When pre-training RNNs in a more ecological task with more than two possible choices, networks develop a strategy by which they gate off the across-trial evidence after errors, mimicking rats′ behavior. Our results suggest that the observed suboptimal behavior reflects the influence of a structural prior that, adaptive in a natural multi-choice environment, constrains performance in a 2AFC laboratory task.

show abstract

Section: Discussionmentioning

confidence: 99%

Ecologically pre-trained RNNs explain suboptimal animal decisions

Molano-Mazón

Duque

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Another set of studies highlighted additional complexity in rodent behavior, as they often switch between states of engagement and disengagement during decision-making tasks 28,29 . These results suggest model-free and inference-based behavior might be interspersed within the same session, potentially engaging different subsets of neural circuits and mechanisms for parallel computation of multiple decision variables 30 . The use of mixture of strategies is further supported by the discovery of separable components of rodent behavior in a reward-guided task 27 .…”

Section: Introductionmentioning

confidence: 94%

Mixture of Learning Strategies Underlies Rodent Behavior in Dynamic Foraging

Yıldırım

Wang

et al. 2022

Preprint

View full text Add to dashboard Cite

In volatile foraging environments, agents need to adapt their learning in accordance with the uncertainty of the environment and knowledge of the hidden structure of the world. In these contexts, previous studies have distinguished between two types of strategies, model-free learning, where reward values are updated locally based on external feedback signals, and inference-based learning, where an internal model of the world is used to make optimal inferences about the current state of the environment. Distinguishing between these strategies during the dynamic foraging behavioral paradigm has been a challenging problem for studies of reward-guided decisions, due to the diversity in behavior of model-free and inference-based agents, as well as the complexities that arise when animals mix between these types of strategies. Here, we developed two solutions that jointly tackle these problems. First, we identified four key behavioral features that together benchmark the switching dynamics of agents in response to a change in reward contingency. We performed computational simulations to systematically measure these features for a large ensemble of model-free and inference-based agents, uncovering an organized structure of behavioral choices where observed behavior can be reliably classified into one of six distinct regimes in the two respective parameter spaces. Second, to address the challenge that arises when animals use multiple strategies within single sessions, we developed a novel state-space method, block Hidden Markov Model (blockHMM), to infer switches in discrete latent states that govern the choice sequences across blocks of trials. Our results revealed a remarkable degree of mixing between different strategies even in expert animals, such that model-free and inference-based learning modes often co-existed within single sessions. Together, these results invite a re-evaluation of the stationarity of behavior during dynamic foraging, provide a comprehensive set of tools to characterize the evolution of learning strategies, and form the basis of understanding neural circuits involved in different modes of behavior within this domain.

show abstract

“…How are higher-order behavioral units, such as sequences and activities, encoded? One possibility is that they could also be encoded in M2 attractors as recent evidence suggests M2 populations encode for a wide range of behavioral variables ( Cazettes et al, 2021 ). We hypothesize an architecture featuring nested assemblies where larger populations, whose activity varies over progressively slower timescales ( Stern et al, 2021 ; Schaub et al, 2015 ), hierarchically encode for slower features of behavior (from actions to sequences to activities).…”

Section: Hierarchical Structurementioning

confidence: 99%

Neural mechanisms underlying the temporal organization of naturalistic animal behavior

Mazzucato

2022

eLife

Self Cite

View full text Add to dashboard Cite

Naturalistic animal behavior exhibits a strikingly complex organization in the temporal domain, with variability arising from at least three sources: hierarchical, contextual, and stochastic. What neural mechanisms and computational principles underlie such intricate temporal features? In this review, we provide a critical assessment of the existing behavioral and neurophysiological evidence for these sources of temporal variability in naturalistic behavior. Recent research converges on an emergent mechanistic theory of temporal variability based on attractor neural networks and metastable dynamics, arising via coordinated interactions between mesoscopic neural circuits. We highlight the crucial role played by structural heterogeneities as well as noise from mesoscopic feedback loops in regulating flexible behavior. We assess the shortcomings and missing links in the current theoretical and experimental literature and propose new directions of investigation to fill these gaps.

show abstract

A repertoire of foraging decision variables in the mouse brain

Cited by 13 publications

References 70 publications

Ecologically pre-trained RNNs explain suboptimal animal decisions

Ecologically pre-trained RNNs explain suboptimal animal decisions

Mixture of Learning Strategies Underlies Rodent Behavior in Dynamic Foraging

Neural mechanisms underlying the temporal organization of naturalistic animal behavior

Contact Info

Product

Resources

About