Multiple Mechanisms for Processing Reward Uncertainty in the Primate Basal Forebrain

Abstract

The ability to use information about the uncertainty of future outcomes is critical for adaptive behavior in an uncertain world. We show that the basal forebrain (BF) contains at least two distinct neural-coding strategies to support this capacity. The dorsal-lateral BF, including the ventral pallidum (VP), contains reward-sensitive neurons, some of which are selectively suppressed by uncertain-reward predictions (U Ϫ ). In contrast, the medial BF (mBF) contains reward-sensitive neurons, some of which are selectively enhanced (U ϩ ) by uncertain-reward predictions. In a two-alternative choice-task, U Ϫ neurons were selectively suppressed while monkeys chose uncertain options over certain options. During the same choice-epoch, U ϩ neurons signaled the subjective reward value of the choice options. Additionally, after the choice was reported, U ϩ neurons signaled reward uncertainty until the choice outcome. We suggest that uncertainty-related suppression of VP may participate in the mediation of uncertainty-seeking actions, whereas uncertainty-related enhancement of the mBF may direct cognitive resources to monitor and learn from uncertain-outcomes.

Novelty and surprise-timing are broadcast by the basal forebrain

“…The tonic activity of these neurons displayed persistent selectivity during the entire 2.5 s CS epoch and was related to the expected values of the CSs. Consistent with previous work (Monosov et al, 2015;Ledbetter et al, 2016), the same neurons displayed ramping activity to the time of uncertain reward delivery or to the delivery of aversive stimuli (Supplemental Figure 9). Here we found that they also ramped to the time of novel stimulus presentation (Supplemental Figure 10) and that their activity is sensitive to both reinforcement deliveries and omissions, signaling the monkeys' surprise (absolute or unsigned reward prediction errors; Figure 2).…”

The Input-Output Relationship of the Cholinergic Basal Forebrain

“…The primate homologue of this area, the anterodorsal septum is known to code for reward uncertainty (Monosov and Hikosaka, 2013), a behavioral measure disrupted in rats following immunolesion of BFc neurons (Cordova and Chiba, 2004, SFN Abstract). Furthermore, reward uncertainty information is carried by basal forebrain units (Ledbetter et al, 2016). This raises the possibility that reward uncertainty is updated in the mPFC with ACh, as driven monosynaptically by the likely GABAergic (Hur and Zaborszky, SFN Abstract, 2007) dorsal lateral (anterodorsal) septum.…”

A neural network for information seeking

“…The standard uncertainty tasks are described in detail in previous work [21][22][23] . They each used a distinct set of fractal visual CSs with different associated outcomes.…”

“…The average responses consisted of sustained ramping to the moment when the uncertain outcome would occur (Fig 1C,D). Importantly, unlike conventional reward-related neurons in these areas [21][22][23] , these neurons were more responsive to reward uncertainty than reward value; their responses were substantially lower for the CSs that cued certain outcomes, even though they had the highest and lowest values in the task (black, 100%, certain reward; gray, 0%, certain noreward; Fig 1C-E). Furthermore, many of these neurons responded to uncertainty in a graded manner 24 : they responded most in the condition with maximal uncertainty (50% reward), less in conditions with intermediate uncertainty (75% and 25% reward), and least in conditions with no uncertainty (100% and 0% reward) 21,22 .…”