Heads for learning, tails for memory: reward, reinforcement and a role of dopamine in determining behavioral relevance across multiple timescales

Abstract

Dopamine has long been tightly associated with aspects of reinforcement learning and motivation in simple situations where there are a limited number of stimuli to guide behavior and constrained range of outcomes. In naturalistic situations, however, there are many potential cues and foraging strategies that could be adopted, and it is critical that animals determine what might be behaviorally relevant in such complex environments. This requires not only detecting discrepancies with what they have recently experienced, but also identifying similarities with past experiences stored in memory. Here, we review what role dopamine might play in determining how and when to learn about the world, and how to develop choice policies appropriate to the situation faced. We discuss evidence that dopamine is shaped by motivation and memory and in turn shapes reward-based memory formation. In particular, we suggest that hippocampal-striatal-dopamine networks may interact to determine how surprising the world is and to either inhibit or promote actions at time of behavioral uncertainty.

Multimodal connectivity‐based parcellation reveals a shell‐core dichotomy of the human nucleus accumbens

“…In general, the differences in the connection patterns provided by the three types of connections roughly supported the concept that the two NAc subregions have different, or even antagonistic, functions [Baudonnat et al, ]. However, these functions of the two subregions must be interrelated and complementary with each other, as indicated by their anatomical and functional interconnections, including direct and indirect connections via multiple circuits.…”

“…The cortical and limbic inputs impinge on and may integrate into the NAc [French and Totterdell, 2003] for the regulation of DA from the MidB [Lodge and Grace, 2006]. DA has been reported to play different roles in the NAc shell and core, involving complementary or even antagonistic functions in some circumstances [Baudonnat et al, 2013]. On the basis of its primary origin from the VTA, the shell was hypothesized to be functionally associated with the mesolimbic dopamine system and to play a critical role in biasing decision-making when a reward is uncertain [Dreyer, 2010;Stopper and Floresco, 2011].…”

Effect of long-term treatment with rasagiline on cognitive deficits and related molecular cascades in aged mice

“…Indeed, selective inhibition of MAO-B does not cause significant changes in central nervous system steadystate levels of 5-HT (Green and Youdim, 1975;Twist et al, 1991;Waldmeier, 1987). In addition, several studies have reported that striatal DA may play a key role in motivated behavior and has a direct impact on memory formation in the hippocampus, suggesting hippocampal-striatal DA network interaction that affected learning and memory capabilities [see reviews, (Shohamy and Adcock, 2010;Baudonnat et al, 2013)]. …”

Burst activation of dopamine neurons produces prolonged post-burst availability of actively released dopamine

“…Dopamine neurons project extensively to striatal and cortical regions where they mediate a diverse array of functions, including reward-related learning, drug-related synaptic plasticity, working memory, and motivation [1][2][3][4][5][6]. Dopamine neurons respond with brief and phasic bursts of action potentials to salient events, such as reward-predictive stimuli and unexpected reward delivery [7,8], and these bursts cause phasic dopamine release in terminal regions, including the prefrontal cortex (PFC) and the nucleus accumbens (NAc) [9][10][11][12].…”