Global reward state affects learning and activity in raphe nucleus and anterior insula in monkeys

Abstract

People and other animals learn the values of choices by observing the contingencies between them and their outcomes. However, decisions are not guided by choice-linked reward associations alone; macaques also maintain a memory of the general, average reward ratethe global reward statein an environment. Remarkably, global reward state affects the way that each choice outcome is valued and influences future decisions so that the impact of both choice success and failure is different in rich and poor environments. Successful choices are more likely to be repeated but this is especially the case in rich environments. Unsuccessful choices are more likely to be abandoned but this is especially likely in poor environments. Functional magnetic resonance imaging (fMRI) revealed two distinct patterns of activity, one in anterior insula and one in the dorsal raphe nucleus, that track global reward state as well as specific outcome events.

Prior cocaine exposure increases firing to immediate reward while attenuating cue and context signals related to reward value in anterior insula

“…We hypothesize that these increases in activity track the overall value of block contexts and contribute to elevating motivation as satiation sets in, based on insula activity’s correlations to changes in both reaction time and percent correct. Such interpretations are in line with a recent publication from Wittman et al (2020), which also implicates insula in tracking long-term reward; their task demonstrates a relationship between BOLD signaling in the macaque insula and task engagement during blocks of trials where probability of reward receipt is overall either higher or lower. Our context effects suggest such global reward signals in insula are encoded at the single-cell level, and may also represent specific reward outcome in addition to reward probability.…”

“…Traditionally, the insula has been described as an interoceptive center (Craig, 2010, 2002; Damasio et al, 2000; Naqvi and Bechara, 2010), but is also thought to contribute to functions related to reward processing and decision-making (Burke and Tobler, 2011; Droutman et al, 2015; Mesulam and Mufson, 1982; Ongur and Price, 2000; Preuschoff et al, 2008; Rogers-Carter and Christianson, 2019). Consistent with these functions, recent work has shown that firing in insula correlates to the anticipation and delivery of both positive and negative outcomes (Guillem et al, 2010; Jo and Jung, 2016; Kusumoto-Yoshida et al, 2015; Mizoguchi et al, 2015; Moschak et al, 2018; Samuelsen et al, 2012; Vincis et al, 2020; Wittmann et al, 2020). Here, we ask if outcome-related neural correlates in insula are disrupted by chronic cocaine-self administration in rats performing a reward-guided decision-making task.…”

“…Insula’s traditional roles integrating experiences like auditory perception (Kimura et al, 2010), gustation (Hanamori et al, 1998), and pain (Downar et al, 2003) with internal state have expanded to reward-guided decision-making. Previous recording studies, for example, demonstrate insula activity encodes anticipation of both positive (Samuelsen et al, 2013, 2012) and negative consequences (Jo and Jung, 2016), and reward consumption (Kusumoto-Yoshida et al, 2015; Wittmann et al, 2020). In relation to its interoceptive functions, insula also seems to track the subjective value of a reward by reflecting how context and experience can change a reward’s value even when the reward itself has not changed.…”

Dissociable mechanisms of reward learning co-mature during human adolescence as predicted by macaque lesion models

“…-how rewarding the environment is overall in the recent past (Wittmann et al, 2020). In a high GRS, macaque monkeys are more likely to stay with a rewarded option while they are more likely to abandon unrewarded choices if the GRS is low.…”

“…By contrast, if the more medial orbitofrontal regions mature faster than the lateral regions, as visually suggested by current developmental maps (Gogtay et al, 2004), then we hypothesize that decision-making processes linked to these regions may not show similar developmental changes during adolescence. We test this hypothesis by examining a type of decision-making bias that occurs in multi-option environments and depends on medial prefrontal cortex: how the comparison of two option is influenced by an irrelevant third alternative (Ray, 1973;Noonan et al, 2010;Louie et al, 2011;Louie et al, 2013;Chau et al, 2014;Noonan et al, 2017;Chau et al, 2020).…”