Complementary Task Structure Representations in Hippocampus and Orbitofrontal Cortex during an Odor Sequence Task

Zhou, Jingfeng; Montesinos-Cartagena, Marlian; Wikenheiser, Andrew M.; Gardner, Michael O.; Niv, Yaron; Schoenbaum, Geoffrey

doi:10.1016/j.cub.2019.08.040

Cited by 50 publications

(48 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Understanding how the brain represents abstract task state-spaces remains a major challenge. Our data joins growing evidence suggesting state-space representations rely on the hippocampal formation 5,9,[34][35][36] and interconnected regions in the ventral PFC 2,3,36,37 . This is particularly interesting in light of the historical role of these regions in generalisation and relational reasoning [38][39][40][41][42] , which are essential for efficient task representations.…”

Section: Discussionsupporting

confidence: 87%

Entorhinal and ventromedial prefrontal cortices abstract and generalise the structure of reinforcement learning problems

Baram

Muller

Nili

et al. 2019

Preprint

View full text Add to dashboard Cite

Knowledge of the structure of a problem, such as relationships between stimuli, enables rapid learning and flexible inference. Humans and other animals can abstract this structural knowledge and generalise it to solve new problems. For example, in spatial reasoning, shortest-path inferences are immediate in new environments. Spatial structural transfer is mediated by grid cells in entorhinal and (in humans) medial prefrontal cortices, which maintain their structure across different environments. Here, using fMRI, we show that entorhinal and mPFC representations perform a much broader role in generalising the structure of problems. We introduce a task-remapping paradigm, where subjects solve multiple reinforcement learning (RL) problems differing in structural or sensory properties. We show that, as with space, entorhinal representations are preserved across different RL problems only if task structure is preserved. In ventromedial PFC, representations of standard RL signals such as prediction error also vary as a function of task structure.

show abstract

Section: Discussionsupporting

confidence: 87%

Entorhinal and ventromedial prefrontal cortices abstract and generalise the structure of reinforcement learning problems

Baram

Muller

Nili

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Understanding how the brain represents abstract task state-spaces remains a major challenge. Our data join growing evidence suggesting state-space representations rely on the hippocampal formation ( Garvert et al., 2017 ; Miller et al., 2017 ; Stachenfeld et al., 2017 ; Vikbladh et al., 2019 ; Wang et al., 2020 ; Zhou et al., 2019b ) and interconnected regions in the ventral PFC ( Schuck et al., 2016 ; Vaidya et al., 2020 ; Wang et al., 2020 ; Wilson et al., 2014 ; Zhou et al., 2019a , 2019b ). This is particularly interesting in light of the historical role of these regions in generalization and relational reasoning ( Barron et al., 2013 ; Bowman and Zeithamova, 2018 ; Cohen and Eichenbaum, 1993 ; Morrissey et al., 2017 ; Preston and Eichenbaum, 2013 ; Zeithamova and Preston, 2010 ), which are essential for efficient task representations.…”

Section: Discussionsupporting

confidence: 85%

Entorhinal and ventromedial prefrontal cortices abstract and generalize the structure of reinforcement learning problems

Baram

Muller

Nili

et al. 2021

Neuron

View full text Add to dashboard Cite

Summary Knowledge of the structure of a problem, such as relationships between stimuli, enables rapid learning and flexible inference. Humans and other animals can abstract this structural knowledge and generalize it to solve new problems. For example, in spatial reasoning, shortest-path inferences are immediate in new environments. Spatial structural transfer is mediated by cells in entorhinal and (in humans) medial prefrontal cortices, which maintain their co-activation structure across different environments and behavioral states. Here, using fMRI, we show that entorhinal and ventromedial prefrontal cortex (vmPFC) representations perform a much broader role in generalizing the structure of problems. We introduce a task-remapping paradigm, where subjects solve multiple reinforcement learning (RL) problems differing in structural or sensory properties. We show that, as with space, entorhinal representations are preserved across different RL problems only if task structure is preserved. In vmPFC and ventral striatum, representations of prediction error also depend on task structure.

show abstract

“…The model further predicts that these experimental results are not simply reflecting memory consolidation of things that have been previously well learned, but are more generally related to the learning process itself. Consistently, the hippocampus has been found to be necessary for learning in several reward-based navigation tasks (Girardeau et al, 2009;Jadhav et al, 2012;de Lavilléon et al, 2015) and interactions between the hippocampus and the orbitofrontal cortex have been recently proposed to contribute to exploit the task structure learned by the internal model to infer the value of stimuli within the environment (Jones et al, 2012;Klein-Flügge et al, 2013;Wikenheiser and Schoenbaum, 2016;Zhou et al, 2019;Park et al, 2019).…”

Section: Discussionmentioning

confidence: 92%

Modeling awake hippocampal reactivations with model-based bidirectional search

Khamassi¹,

Girard²

2020

Biol Cybern

View full text Add to dashboard Cite

Hippocampal offline reactivations during reward-based learning, usually categorized as replay events, have been found to be important for performance improvement over time and for memory consolidation. Recent computational work has linked these phenomena to the need to transform reward information into stateaction values for decision-making and to propagate it to all relevant states of the environment. Nevertheless, it is still unclear whether an integrated reinforcement learning mechanism could account for the variety of awake hippocampal reactivations, including variety in order (forward and reverse reactivated trajectories) and variety in the location where they occur (reward site or decision-point). Here we present a model-based bidirectional search model which accounts for a variety of hippocampal reactivations. The model combines forward trajectory sampling from current position and backward sampling through prioritized sweeping from states associated with large reward prediction errors until the two trajectories connect. This is repeated until stabilization of state-action values (convergence), which could explain why hippocampal reactivations drastically diminish when the animal's performance stabilizes. Simulations in a multiple T-maze task show that forward reactivations are prominently found at decision-points while backward reactivations are exclusively generated at reward sites. Finally, the model can generate imaginary trajectories that are not allowed to the agent during task performance. We raise some experimental predictions and implications for future studies of the role of the hippocampo-prefronto-striatal network in learning.

show abstract

Complementary Task Structure Representations in Hippocampus and Orbitofrontal Cortex during an Odor Sequence Task

Cited by 50 publications

References 43 publications

Entorhinal and ventromedial prefrontal cortices abstract and generalise the structure of reinforcement learning problems

Entorhinal and ventromedial prefrontal cortices abstract and generalise the structure of reinforcement learning problems

Entorhinal and ventromedial prefrontal cortices abstract and generalize the structure of reinforcement learning problems

Modeling awake hippocampal reactivations with model-based bidirectional search

Contact Info

Product

Resources

About