Hippocampus Leads Ventral Striatum in Replay of Place-Reward Information

Lansink, Carien S.; Goltstein, Pieter M.; Lankelma, J.; McNaughton, Bruce L.; Pennartz, Cyriel M. A.

doi:10.1371/journal.pbio.1000173

Cited by 368 publications

(372 citation statements)

References 55 publications

(88 reference statements)

Supporting

Mentioning

334

Contrasting

Order By: Relevance

“…Here we link epistemic actions to a form of "mental" exploration, in which episodic content is retrieved from memory (Eichenbaum 2000;Tulving 1972) or simulated/recombined in a constructive way to construct novel episodes (Hassabis and Maguire 2009;, thus leaving open the possibility that VTE-based declarative control engages a mixture of model based and episodic components. In turn, the episodic component can include a mixture of contextual, spatial, and reward information (e.g., through reactivations of hippocampus and ventral striatum, Lansink et al 2009). The engagement of epistemic behavior does not require any special arbitration; it is a natural consequence of casting the utility of a reward in terms of a prior belief, which enables it to be compared directly with the epistemic value of (overt and covert) exploratory behavior.…”

Section: Discussionmentioning

confidence: 99%

“…Simultaneously, covert expectations of rewards can be measured in the ventral striatum van der Meer et al 2012). The hippocampus-ventral striatum pathway might represent a neurophysiological mechanism for VTE-based spatial deliberation and search processes, where memories of past experiences, associated with each choice, are recalled-or future events are simulated based on episodic memories-and used off-line to update the subjective value of the subsequent choice and select among plans (Lansink et al 2009;van der Meer and Redish 2010;Pennartz et al 2011;Pezzulo et al 2013Pezzulo et al , 2014.In sum, VTE behavior and associated hippocampal forward sweeps have been linked to deliberation and the (covert) evaluation of choice alternatives. They are an ideal paradigm for studying the various trade-offs between habitual and deliberate forms of choice.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Active Inference, epistemic value, and vicarious trial and error

et al. 2016

View full text Add to dashboard Cite

Balancing habitual and deliberate forms of choice entails a comparison of their respective merits-the former being faster but inflexible, and the latter slower but more versatile. Here, we show that arbitration between these two forms of control can be derived from first principles within an Active Inference scheme. We illustrate our arguments with simulations that reproduce rodent spatial decisions in T-mazes. In this context, deliberation has been associated with vicarious trial and error (VTE) behavior (i.e., the fact that rodents sometimes stop at decision points as if deliberating between choice alternatives), whose neurophysiological correlates are "forward sweeps" of hippocampal place cells in the arms of the maze under consideration. Crucially, forward sweeps arise early in learning and disappear shortly after, marking a transition from deliberative to habitual choice. Our simulations show that this transition emerges as the optimal solution to the trade-off between policies that maximize reward or extrinsic value (habitual policies) and those that also consider the epistemic value of exploratory behavior (deliberative or epistemic policies)-the latter requiring VTE and the retrieval of episodic information via forward sweeps. We thus offer a novel perspective on the optimality principles that engender forward sweeps and VTE, and on their role on deliberate choice.Substantial evidence indicates that animal behavior is determined both by deliberative processes (i.e., based on predictions of future outcomes and rewards) and by habitual reflexes (i.e., based on stimulus-response associations; Balleine and Dickinson 1998). The former are more resource intensive and sensitive to changes in task contingencies, while the latter are cheaper but inflexible; hence whether it is optimal to call on deliberative or habitual choice depends on the trade-off between the advantage of flexibility and computational costs (Balleine and Dickinson 1998;Dolan and Dayan 2013;Lee et al. 2014). In this paper, we try to understand the contextualization of behavior and the trade-off between deliberative and habitual choice from first principles, using Active Inference and Markov decision process models of exploitation and exploration (Friston et al. 2013(Friston et al. , 2014Pezzulo et al. 2015).We focus specifically on vicarious trial and error (VTE) behavior, which is considered a hallmark of deliberation (Muenzinger 1938;Tolman 1938Tolman , 1939. This is based on the observation that, when rodents have to remember or search the correct route to a reward in a maze (e.g., a T-maze), they sometimes stop at choice points, to look left and right before choosing which direction to go. This has been interpreted as a signature of cognitive search and deliberation between the two choices (i.e., going right or left). In keeping with a role of VTE behavior for deliberation, it occurs early in learning and decreases or disappears after significant experience (Tolman 1939;van der Meer and Redish 2010;van der Meer et al. 2012) but it can incr...

show abstract

Section: Discussionmentioning

confidence: 99%

mentioning

confidence: 99%

Active Inference, epistemic value, and vicarious trial and error

et al. 2016

View full text Add to dashboard Cite

show abstract

“…In particular, a spontaneous reactivation (or replay) of neuronal firing patterns occurs in the ventral striatum of rats after a reward searching behavior (Lansink et al 2008;Pennartz et al 2004). This off-line replay is induced by hippocampal ripples and seems to be related to linking a memory trace to a motivational or emotional value during sleep (Lansink et al 2009). Whether a similar neural mechanism exists in humans is still unclear, but brain imaging studies have demonstrated that the hippocampus and the ventral striatum are also activated during NREM sleep (Nofzinger et al 2002), and that increased hippocampal activity may relate to subsequent performance improvement (Peigneux et al 2004).…”

Section: Activation Of Reward Networkmentioning

confidence: 99%

“…Below we report converging evidence suggesting that the activation of limbic, paralimbic, and reward systems during sleep-with or without concomitant dream experience-serves emotional regulation and memory consolidation processes. During NREM sleep, activation of the hippocampus and ventral striatum enables the formation of a memory trace comprising contextual, emotional, and motivational components (Lansink et al 2009). The coordinated reactivation of both the hippocampus and the ventral striatum during SWS provides a possible mechanism for the consolidation of associative memory-reward information (Fig.…”

Section: Off-line Processing and Consolidation Of Emotional Memoriesmentioning

confidence: 99%

Sleep and Emotional Functions

Perogamvros

Schwartz

2013

Sleep, Neuronal Plasticity and Brain Function

View full text Add to dashboard Cite

In this chapter, we review studies investigating the role of sleep in emotional functions. In particular, evidence has recently accumulated to show that brain regions involved in the processing of emotional and reward-related information are activated during sleep. We suggest that such activation of emotional and reward systems during sleep underlies the reprocessing and consolidation of memories with a high affective and motivational relevance for the organism. We also propose that these mechanisms occurring during sleep promote adapted cognitive and emotional responses in the waking state, including overnight performance improvement, creativity, and sexual functions. Activation across emotional-limbic circuits during sleep also appears to promote emotional maturation and the emergence of consciousness in the developing brain.

show abstract

“…However, the hippocampal formation sends projections to several structures, including prefrontal cortex (Swanson, 1981) and the ventral striatum (VS) (Groenewegen et al, 1987). Both structures exhibit coherent theta rhythmicity (Tabuchi et al, 2000;Martin, 2001;Jones and Wilson, 2005a;Siapas et al, 2005;Adhikari et al, 2010), theta phase locking (Jones and Wilson, 2005b;Benchenane et al, 2010;Hyman et al, 2010), and coordinated "replay" with the hippocampus (HC) (Lansink et al, 2009;Peyrache et al, 2009;Benchenane et al, 2010), suggesting a relationship to hippocampal temporal coding schemes. In general, the hippocampus-ventral striatal projection has long been implicated in the modulation of motivated behavior (Mogenson et al, 1980;Pennartz et al, 1994;Bast and Feldon, 2003;.…”

Section: Introductionmentioning

confidence: 99%

Theta Phase Precession in Rat Ventral Striatum Links Place and Reward Information

Meer¹,

Redish²

2011

J. Neurosci.

208

192

View full text Add to dashboard Cite

A functional interaction between the hippocampal formation and the ventral striatum is thought to contribute to the learning and expression of associations between places and rewards. However, the mechanism of how such associations may be learned and used is currently unknown. We recorded neural ensembles and local field potentials from the ventral striatum and CA1 simultaneously as rats ran a modified T-maze. Theta-modulated cells in ventral striatum almost invariably showed firing phase precession relative to the hippocampal theta rhythm. Across the population of ventral striatal cells, phase precession was preferentially associated with an anticipatory ramping of activity up to the reward sites. In contrast, CA1 population activity and phase precession were distributed more uniformly. Ventral striatal phase precession was stronger to hippocampal than ventral striatal theta and was accompanied by increased theta coherence with hippocampus, suggesting that this effect is hippocampally derived. These results suggest that the firing phase of ventral striatal neurons contains motivationally relevant information and that phase precession serves to bind hippocampal place representations to ventral striatal representations of reward.

show abstract

Hippocampus Leads Ventral Striatum in Replay of Place-Reward Information

Abstract: While the brain is asleep, the hippocampus plays first fiddle in the orchestra of memory; spatial information is reactivated in the hippocampus shortly in advance of emotional memory traces in the vental striatum.

Cited by 368 publications

References 55 publications

Active Inference, epistemic value, and vicarious trial and error

Active Inference, epistemic value, and vicarious trial and error

Sleep and Emotional Functions

Theta Phase Precession in Rat Ventral Striatum Links Place and Reward Information

Contact Info

Product

Resources

About