Hippocampal Neural Correlates for Values of Experienced Events

Huh

et al. 2017

Front. Neural Circuits

We have shown previously that CA1 conveys significant neural signals necessary to update value of the chosen target, namely chosen value and reward signals. To better understand hippocampal neural processes related to valuation, we compared chosen value- and reward-related neural activity between the CA3 and CA1 regions. Single units were recorded with tetrodes from the dorsal CA3 and CA1 regions of rats performing a dynamic foraging task, and chosen value- and reward-related neural activity was estimated using a reinforcement learning model and multiple regression analyses. Neural signals for chosen value and reward converged in both CA3 and CA1 when a trial outcome was revealed. However, these neural signals were stronger in CA1 than CA3. Consequently, neural signals for reward prediction error and updated chosen value were stronger in CA1 than CA3. Together with our previous finding that CA1 conveys stronger value signals than the subiculum, our results raise the possibility that CA1 might play a particularly important role among hippocampal subregions in evaluating experienced events.

Section: Resultssupporting

confidence: 71%

Neural Signals Related to Outcome Evaluation Are Stronger in CA1 than CA3

Huh

et al. 2017

Front. Neural Circuits

Journal of Neurophysiology

“…Furthermore, chosen value cells have no known correspondent in perceptual decision tasks. In contrast, chosen value signals have been observed in numerous studies of value-based decisions and in multiple brain regions (Amemori and Graybiel 2012;Cai et al 2011;Padoa-Schioppa 2012, 2014;Grabenhorst et al 2012;Lau and Glimcher 2008;Lee et al 2012;Padoa-Schioppa and Assad 2006;Roesch et al 2009;Strait et al 2014;Sul et al 2010;Wunderlich et al 2010). While it is clear that chosen value signals could inform a variety of mental functions including associative learning, visual attention, emotion, and others, the possible contributions of chosen value signals to economic decisions have remained mysterious.…”

Section: Discussionmentioning

confidence: 99%

A neuro-computational model of economic decisions

Rustichini

Padoa-Schioppa

2015

107

1314 -1328, 2006). The domain of economic decisions is significantly broader than that for which the model was originally designed, yet the model performed remarkably well. The input and output nodes of the network were naturally mapped onto two groups of cells in OFC. Surprisingly, the activity of interneurons in the network closely resembled that of the third group of cells, namely, chosen value cells. The model reproduced several phenomena related to the neuronal origins of choice variability. It also generated testable predictions on the excitatory/inhibitory nature of different neuronal populations and on their connectivity. Some aspects of the empirical data were not reproduced, but simple extensions of the model could overcome these limitations. These results render a biologically credible model for the neuronal mechanisms of economic decisions. They demonstrate that choices could emerge from the activity of cells in the OFC, suggesting that chosen value cells directly participate in the decision process. Importantly, Wang's model provides a platform to investigate the implications of neuroscience results for economic theory.

Neurobiology of Learning and Memory

“…Thus, the finding that generalization in acquired equivalence predicts model-based learning suggests hippocampal involvement also in the latter function. Although the hippocampus surfaces relatively rarely in the literature on RL and decision making (compared to striatal and ventromedial prefrontal areas), it has been implicated in several studies with elaborated task preparations (Tanaka et al, 2004; Lee et al, 2012; Wimmer et al, 2012; Wimmer and Shohamy, 2012), including those formally examining model-based decisions (Simon and Daw, 2011; Bornstein and Daw, In Press). Moreover, hippocampal involvement is plausible in light of the broader pattern of previous results, which implicate it both in representing models and utilizing those representations for planning.…”

Section: Discussionmentioning

confidence: 99%

Multiple memory systems as substrates for multiple decision systems

Doll

Shohamy

Daw

2015

101

It has recently become widely appreciated that value-based decision making is supported by multiple computational strategies. In particular, animal and human behavior in learning tasks appears to include habitual responses described by prominent model-free reinforcement learning (RL) theories, but also more deliberative or goal-directed actions that can be characterized by a different class of theories, model-based RL. The latter theories evaluate actions by using a representation of the contingencies of the task (as with a learned map of a spatial maze), called an “internal model.” Given the evidence of behavioral and neural dissociations between these approaches, they are often characterized as dissociable learning systems, though they likely interact and share common mechanisms. In many respects, this division parallels a longstanding dissociation in cognitive neuroscience between multiple memory systems, describing, at the broadest level, separate systems for declarative and procedural learning. Procedural learning has notable parallels with model-free RL: both involve learning of habits and both are known to depend on parts of the striatum. Declarative memory, by contrast, supports memory for single events or episodes and depends on the hippocampus. The hippocampus is thought to support declarative memory by encoding temporal and spatial relations among stimuli and thus is often referred to as a relational memory system. Such relational encoding is likely to play an important role in learning an internal model, the representation that is central to model-based RL. Thus, insofar as the memory systems represent more general-purpose cognitive mechanisms that might subserve performance on many sorts of tasks including decision making, these parallels raise the question whether the multiple decision systems are served by multiple memory systems, such that one dissociation is grounded in the other. Here we investigated the relationship between model-based RL and relational memory by comparing individual differences across behavioral tasks designed to measure either capacity. Human subjects performed two tasks, a learning and generalization task (acquired equivalence) which involves relational encoding and depends on the hippocampus; and a sequential RL task that could be solved by either a model-based or model-free strategy. We assessed the correlation between subjects’ use of flexible, relational memory, as measured by generalization in the acquired equivalence task, and their differential reliance on either RL strategy in the decision task. We observed a significant positive relationship between generalization and model-based, but not model-free, choice strategies. These results are consistent with the hypothesis that model-based RL, like acquired equivalence, relies on a more general-purpose relational memory system.