Basal ganglia engagement during feedback processing after a substantial delay

Abstract

The striatum has been shown to play an important role in learning from performance-related feedback that is presented shortly after each response. However, less is known about the neural mechanisms supporting learning from feedback that is substantially delayed from the original response. Since the consequences of one’s actions often do not become known until after a delay, it is important to understand whether delayed feedback can produce neural responses similar to those elicited by immediate feedback presentation. We investigated this issue by using functional magnetic resonance imaging (fMRI) as participants performed a paired-associate learning task with 180 distinct trials. Feedback indicating response accuracy was presented immediately, after a delay of 25 minutes, or not at all. Both immediate and delayed feedback led to significant gains in accuracy on a post-test, relative to no feedback. Replicating previous work, we found that the caudate nuclei showed greater activation for positive feedback than negative feedback when the feedback was presented immediately. In addition, delayed feedback also led to differential caudate activity to positive versus negative feedback. Delayed negative feedback also produced significant activation of the putamen and globus pallidus (the lentiform nucleus), relative to no feedback and delayed positive feedback. This suggests that the caudate nucleus is sensitive to the affective nature of feedback, across different timescales, while the lentiform nucleus may be particularly involved in processing the information carried by negative feedback after a substantial delay.

Modulation of ventral striatal activity by cognitive effort

“…In addition, two conditions in the current experiment did not require any cognitive effort and resulted in random feedback presentation, thus providing a visuo-motor control for the 1- and 2-step conditions. Consistent with previous findings (Elliott et al ., 1997; Delgado et al ., 2000; Tricomi et al ., 2004, 2006; Tricomi & Fiez, 2008; Dobryakova & Tricomi, 2013; Sescousse et al ., 2013; DePasque Swanson & Tricomi, 2014), we observed a robust main effect of valence in the dorsal and ventral striatum, driven by significant differences between positive and negative feedback presentation in all four conditions. Furthermore, in the same voxels that showed a sensitivity to positive versus negative feedback, we also observed enhanced VS activation in association with the more difficult and cognitively effortful condition, i.e., the feedback presentation during the 2-step learning condition.…”

“…The reward experienced when a goal is achieved depends on the value one places on the goal. The ventral striatum (VS) has been shown to play an important role in processing goal values, both for extrinsic, or tangible, outcomes (such as food rewards and monetary gain or loss) (Knutson et al ., 2001; Kurniawan et al ., 2013; Tricomi & Lempert, 2015) and for intrinsic, or nontangible, outcomes (such as positive and negative feedback during learning) (Ullsperger & von Cramon, 2003; Lutz et al ., 2012; Dobryakova & Tricomi, 2013; DePasque Swanson & Tricomi, 2014). …”

Contributions of the hippocampus to feedback learning

“…Hippocampal signals have been shown to differentiate between correct and incorrect feedback during some forms of probabilistic, trial-and-error learning (Dobryakova & Tricomi, 2013; Li, Delgado, & Phelps, 2011)—a pattern of activation similar to responses observed in the striatum (Delgado, 2007; Delgado et al 2000; Tricomi et al 2004). This is consistent with processing a prediction error learning signal—a signal that indicates whether the outcome was better or worse than expected—which typically is associated with blood-oxygen-level-dependent (BOLD) responses in the striatum in humans (for reviews, see Daw & Doya, 2006; O’Doherty, 2004) and with dopamine midbrain neuronal activity in nonhuman primates (Schultz, Dayan, & Montague, 1997).…”

“…However, MTL activity is not always observed during feedback learning (Delgado, Miller, Inati, & Phelps, 2005; Delgado, Nystrom, Fissell, Noll, & Fiez, 2000; Dobryakova & Tricomi, 2013; Shohamy et al 2004a; Tricomi, Delgado, & Fiez, 2004; Wilkinson et al, 2014)—a process that seems to be more reliant on the integrity of the BG as evidenced by difficulties in learning via feedback expressed by patients with Parkinson’s disease (Foerde, Braun, & Shohamy, 2013a; Foerde, Race, et al, 2013b; Foerde & Shohamy, 2011a, b; Jahanshahi, Wilkinson, Gahir, Dharminda, & Lagnado, 2010; Knowlton, Mangels, & Squire, 1996; Myers et al 2003; Shohamy et al 2004a, b; Wilkinson, Lagnado, Quallo, & Jahanshahi, 2008), but not with patients with amnesia (Foerde, Race, et al, 2013b; Knowlton et al 1996; Knowlton, Squire, & Gluck, 1994; Myers et al 2003). Given the conflicting results and differences across paradigms, it is unclear what aspects of feedback learning engage the MTL, particularly the hippocampus.…”

“…Indeed, patients with amnesia are impaired in probabilistic learning when the feedback is delayed by 7 s (Foerde, Race, et al, 2013b), and activation of the hippocampus in an fMRI study is observed with a similar delay (Foerde & Shohamy, 2011a). In addition, the length of the feedback delay may negatively impact recruitment of the hippocampus (Dobryakova & Tricomi, 2013), highlighting how sensitive the involvement of the hippocampus is during feedback learning.…”