Determining the Neural Substrates of Goal-Directed Learning in the Human Brain

Abstract

Instrumental conditioning is considered to involve at least two distinct learning systems: a goal-directed system that learns associations between responses and the incentive value of outcomes, and a habit system that learns associations between stimuli and responses without any link to the outcome that that response engendered. Lesion studies in rodents suggest that these two distinct components of instrumental conditioning may be mediated by anatomically distinct neural systems. The aim of the present study was to determine the neural substrates of the goal-directed component of instrumental learning in humans. Nineteen human subjects were scanned with functional magnetic resonance imaging while they learned to choose instrumental actions that were associated with the subsequent delivery of different food rewards (tomato juice, chocolate milk, and orange juice). After training, one of these foods was devalued by feeding the subject to satiety on that food. The subjects were then scanned again, while being re-exposed to the instrumental choice procedure (in extinction). We hypothesized that regions of the brain involved in goal-directed learning would show changes in their activity as a function of outcome devaluation. Our results indicate that neural activity in one brain region in particular, the orbitofrontal cortex, showed a strong modulation in its activity during selection of a devalued compared with a nondevalued action. These results suggest an important contribution of orbitofrontal cortex in guiding goal-directed instrumental choices in humans.

The role of learning-related dopamine signals in addiction vulnerability

“…Similar findings hold in closely related paradigms in humans (Valentin et al, 2007;de Wit et al, 2009;Tricomi et al, 2009;Daw et al, 2011). The shift from early devaluation sensitivity to later devaluation insensitivity can be explained by the statistical properties of model-based and model-free systems, respectively.…”

“…In other words, the information reflected in the habitual behaviour is present in a stimulus-action value Q MF (s, a), which captures how valuable an action is in a particular state, but without providing any information about the actual outcomes. An insensitivity to motivational changes is characteristic of cached values and habitual choices (McClure et al, 2003b;Daw et al, 2005;Valentin et al, 2007;Daw et al, 2011;Wunderlich et al, 2012a). Thus, instrumental learning derived from the accumulation of dopaminergic prediction errors accounts for outcome-insensitive habits.…”

Dopamine Agonists Diminish Value Sensitivity of the Orbitofrontal Cortex: A Trigger for Pathological Gambling in Parkinson's Disease?

“…In the addict, the value that is attributed to certain events or cues seems to be altered (Garavan et al, 2000;Ahmed et al, 2002;Grigson and Twining, 2002). There is substantial evidence that the OFC mediates subjective value attribution and is an integral part in adaptive decision making (Tremblay and Schultz, 1999;Knutson et al, 2000;Breiter et al, 2001;Elliott et al, 2003;Valentin et al, 2007). Indeed, a recent activation study in cocaine users confirmed the involvement of the lateral OFC in deficient attribution of feedback values (Goldstein et al, 2007).…”

Where is the chocolate? Rapid spatial orienting toward stimuli associated with primary rewards

“…Similarly, the more they judged the chocolate odor as pleasant (r = .64, p < .001,), the more they preferred the CS+ compared with the CSÀ and the baseline. (2008): We selected three geometric complex figures typically used in human conditioning paradigms (Gottfried et al, 2003;O'Doherty et al, 2004;Valentin, Dickinson, & O'Doherty, 2007) that in a pilot study (n = 26) were rated as equally neutral on a pleasantness scale. During ''task-on'' periods, the CS+ or CSÀ was displayed, a target appeared at the center of the image, and participants had to press a keyboard that triggers odor release.…”