Neural Components Underlying Behavioral Flexibility in Human Reversal Learning

Abstract

The ability to flexibly respond to changes in the environment is critical for adaptive behavior. Reversal learning (RL) procedures test adaptive response updating when contingencies are altered. We used functional magnetic resonance imaging to examine brain areas that support specific RL components. We compared neural responses to RL and initial learning (acquisition) to isolate reversal-related brain activation independent of cognitive control processes invoked during initial feedback-based learning. Lateral orbitofrontal cortex (OFC) was more activated during reversal than acquisition, suggesting its relevance for reformation of established stimulus-response associations. In addition, the dorsal anterior cingulate (dACC) and right inferior frontal gyrus (rIFG) correlated with change in postreversal accuracy. Because optimal RL likely requires suppression of a prior learned response, we hypothesized that similar regions serve both response inhibition (RI) and inhibition of learned associations during reversal. However, reversal-specific responding and stopping (requiring RI and assessed via the stop-signal task) revealed distinct frontal regions. Although RI-related regions do not appear to support inhibition of prepotent learned associations, a subset of these regions, dACC and rIFG, guide actions consistent with current reward contingencies. These regions and lateral OFC represent distinct neural components that support behavioral flexibility important for adaptive learning.

Neural changes when actions change: Adaptation of strong and weak expectations

“…In the balanced states, evidence for neither internal model outweighs evidence for the other and suppression could be regarded as too persistent (for the divergent internal model) or too premature (for the original internal model), respectively. Activity of the OFC in the interaction contrast complements the emerging picture (Ghahremani, et al, 2009). Biased states necessarily have one strong, or solid component, like a prepotent response or well practiced forward model.…”

“…Both, hypothesis generation and suppression have been discussed as potential OFC functions. (Elliott, Dolan, & Frith, 2000;Ghahremani, et al, 2009;Vartanian & Goel, 2005). Hypothesis generation and suppression can be reframed as evaluation or weight changes as a results of evaluation, which itself is a function ascribed to the OFC (Wallis, 2007).…”

“…However, it remains to be experimentally established how an internal model's solidity influences its revision and hence adaptation of predictions. To our knowledge only a few studies on reversal learning in stimulus-response paradigms (Ghahremani, Monterosso, Jentsch, Bilder, & Poldrack, 2009) have dealt with the influence of model solidity on adaptation; no study has addressed the question in an action observation paradigm.…”

Effect of Modafinil on Learning and Task-Related Brain Activity in Methamphetamine-Dependent and Healthy Individuals

“…During scanning, participants performed a deterministic associative learning task with reversal components, which required selection of the correct response between two possible responses associated with a picture of an abstract visual pattern (Ghahremani et al, 2010) (Figure 1a). On each trial, participants were presented with a picture and were asked to decide whether it was associated with a left or right key response.…”

“…In a randomized, within-subject, placebocontrolled, double-blind crossover design, participants performed an associative learning task while undergoing functional magnetic resonance imaging (fMRI). The associative learning task was selected on the basis of prior work showing that it elicits fMRI activation in neural circuitry important for learning (Ghahremani et al, 2010). Moreover, as preliminary findings indicated that performance on this task was worse in MA-dependent individuals than in healthy control subjects, and negatively correlated with dopamine D2/D3 receptor availability (Ghahremani et al, 2009), we expected modafinil to improve performance in MA-dependent individuals, given evidence that it can enhance dopaminergic function (Madras et al, 2006;Volkow et al, 2009).…”