Interactions between deliberation and delay-discounting in rats

Papale, Andrew E.; Stott, Jeffrey J.; Powell, Nathaniel J.; Regier, Paul S.; Redish, A. David

doi:10.3758/s13415-012-0097-7

Cited by 73 publications

(131 citation statements)

References 54 publications

(72 reference statements)

Supporting

Mentioning

110

Contrasting

Order By: Relevance

“…This behavioral phenomenon is termed vicarious trial-and-error (VTE) behavior and has been hypothesized to reflect an underlying search process Muenzinger 1938;Tolman 1938). VTE primarily occurs during goal-directed behaviors (Gardner et al 2013;Papale et al 2012;Schmidt et al 2013;Tolman 1938).…”

mentioning

confidence: 99%

Hippocampus and subregions of the dorsal striatum respond differently to a behavioral strategy change on a spatial navigation task

Regier

Amemiya

Redish

2015

Journal of Neurophysiology

Self Cite

View full text Add to dashboard Cite

Regier PS, Amemiya S, Redish AD. Hippocampus and subregions of the dorsal striatum respond differently to a behavioral strategy change on a spatial navigation task. J Neurophysiol 114: 1399 -1416, 2015. First published June 18, 2015 doi:10.1152/jn.00189.2015.-Goal-directed and habit-based behaviors are driven by multiple but dissociable decision making systems involving several different brain areas, including the hippocampus and dorsal striatum. On repetitive tasks, behavior transitions from goal directed to habit based with experience. Hippocampus has been implicated in initial learning and dorsal striatum in automating behavior, but recent studies suggest that subregions within the dorsal striatum have distinct roles in mediating habit-based and goal-directed behavior. We compared neural activity in the CA1 region of hippocampus with anterior dorsolateral and posterior dorsomedial striatum in rats on a spatial choice task, in which subjects experienced reward delivery changes that forced them to adjust their behavioral strategy. Our results confirm the importance of the hippocampus in evaluating predictive steps during goal-directed behavior, while separate circuits in the basal ganglia integrated relevant information during automation of actions and recognized when new behaviors were needed to continue obtaining rewards.hippocampus; navigation; neural ensemble; striatum; tetrode THE PROCESS OF LEARNING and automating actions is driven by multiple but dissociable neural circuits that instantiate different decision making systems (Balleine et al.

show abstract

mentioning

confidence: 99%

Hippocampus and subregions of the dorsal striatum respond differently to a behavioral strategy change on a spatial navigation task

Regier

Amemiya

Redish

2015

Journal of Neurophysiology

Self Cite

View full text Add to dashboard Cite

show abstract

“…Although this situation does not induce higher contextual uncertainty than the previous transition (from left-right to leftleft), it requires a more flexible change of strategy-as one cannot keep executing the first (left) action as in the previous case. It is this case that demands more forward sweeps, in keeping with a recognized role of VTE behavior in promoting behavioral flexibility (Papale et al 2012). Here, the increased number of VTE events is plausibly due to the fact that, after the first (left) choice, the animal faces the impossibility to pursue its policy-as the path is blocked.…”

Section: Discussionmentioning

confidence: 99%

Active Inference, epistemic value, and vicarious trial and error

et al. 2016

View full text Add to dashboard Cite

Balancing habitual and deliberate forms of choice entails a comparison of their respective merits-the former being faster but inflexible, and the latter slower but more versatile. Here, we show that arbitration between these two forms of control can be derived from first principles within an Active Inference scheme. We illustrate our arguments with simulations that reproduce rodent spatial decisions in T-mazes. In this context, deliberation has been associated with vicarious trial and error (VTE) behavior (i.e., the fact that rodents sometimes stop at decision points as if deliberating between choice alternatives), whose neurophysiological correlates are "forward sweeps" of hippocampal place cells in the arms of the maze under consideration. Crucially, forward sweeps arise early in learning and disappear shortly after, marking a transition from deliberative to habitual choice. Our simulations show that this transition emerges as the optimal solution to the trade-off between policies that maximize reward or extrinsic value (habitual policies) and those that also consider the epistemic value of exploratory behavior (deliberative or epistemic policies)-the latter requiring VTE and the retrieval of episodic information via forward sweeps. We thus offer a novel perspective on the optimality principles that engender forward sweeps and VTE, and on their role on deliberate choice.Substantial evidence indicates that animal behavior is determined both by deliberative processes (i.e., based on predictions of future outcomes and rewards) and by habitual reflexes (i.e., based on stimulus-response associations; Balleine and Dickinson 1998). The former are more resource intensive and sensitive to changes in task contingencies, while the latter are cheaper but inflexible; hence whether it is optimal to call on deliberative or habitual choice depends on the trade-off between the advantage of flexibility and computational costs (Balleine and Dickinson 1998;Dolan and Dayan 2013;Lee et al. 2014). In this paper, we try to understand the contextualization of behavior and the trade-off between deliberative and habitual choice from first principles, using Active Inference and Markov decision process models of exploitation and exploration (Friston et al. 2013(Friston et al. , 2014Pezzulo et al. 2015).We focus specifically on vicarious trial and error (VTE) behavior, which is considered a hallmark of deliberation (Muenzinger 1938;Tolman 1938Tolman , 1939. This is based on the observation that, when rodents have to remember or search the correct route to a reward in a maze (e.g., a T-maze), they sometimes stop at choice points, to look left and right before choosing which direction to go. This has been interpreted as a signature of cognitive search and deliberation between the two choices (i.e., going right or left). In keeping with a role of VTE behavior for deliberation, it occurs early in learning and decreases or disappears after significant experience (Tolman 1939;van der Meer and Redish 2010;van der Meer et al. 2012) but it can incr...

show abstract

“…As a rat reaches a choice point it frequently pauses and orients toward potential routes of travel, a process termed vicarious trial and error (VTE) (Muenzinger and Gentry 1931;Muenzinger 1938;Tolman 1938). VTE behavior was quantified by the IdPhi measure, the integrated absolute change in angular displacement of the rat's head (Papale et al 2012;see Materials and Methods). Theoretically, the IdPhi distribution should be positively skewed ( Fig.…”

Section: Resultsmentioning

confidence: 99%

“…VTE was preferentially seen at the highcost choice point on the Multiple-T-Maze task (Johnson and Redish 2007;van der Meer and Redish 2010;Blumenthal et al 2011). VTE was seen at the choice point on a spatial delay discounting task, a task where rats had to choose between a smaller immediate reward and a larger delayed reward (Papale et al 2012). Theorists posit that VTE occurs in behavioral paradigms with changing task demands (Johnson et al 2012).…”

Section: Vicarious Trial and Error: Strategy Conflict/competitionmentioning

confidence: 99%

See 1 more Smart Citation

Conflict between place and response navigation strategies: Effects on vicarious trial and error (VTE) behaviors

et al. 2013

Self Cite

View full text Add to dashboard Cite

Navigation can be accomplished through multiple decision-making strategies, using different information-processing computations. A well-studied dichotomy in these decision-making strategies compares hippocampal-dependent "place" and dorsal-lateral striatal-dependent "response" strategies. A place strategy depends on the ability to flexibly respond to environmental cues, while a response strategy depends on the ability to quickly recognize and react to situations with welllearned action -outcome relationships. When rats reach decision points, they sometimes pause and orient toward the potential routes of travel, a process termed vicarious trial and error (VTE). VTE co-occurs with neurophysiological information processing, including sweeps of representation ahead of the animal in the hippocampus and transient representations of reward in the ventral striatum and orbitofrontal cortex. To examine the relationship between VTE and the place/response strategy dichotomy, we analyzed data in which rats were cued to switch between place and response strategies on a plus maze. The configuration of the maze allowed for place and response strategies to work competitively or cooperatively. Animals showed increased VTE on trials entailing competition between navigational systems, linking VTE with deliberative decision-making. Even in a well-learned task, VTE was preferentially exhibited when a spatial selection was required, further linking VTE behavior with decision-making associated with hippocampal processing.

show abstract

Interactions between deliberation and delay-discounting in rats

Cited by 73 publications

References 54 publications

Hippocampus and subregions of the dorsal striatum respond differently to a behavioral strategy change on a spatial navigation task

Hippocampus and subregions of the dorsal striatum respond differently to a behavioral strategy change on a spatial navigation task

Active Inference, epistemic value, and vicarious trial and error

Conflict between place and response navigation strategies: Effects on vicarious trial and error (VTE) behaviors

Contact Info

Product

Resources

About