Changes in response contingencies require adjusting ones assumptions about outcomes of behaviors. Such adaptation processes are driven by reward prediction error (RPE) signals which reflect the inadequacy of expectations. Signals resembling RPEs are known to be encoded by mesencephalic dopamine neurons projecting to the striatum and frontal regions. Although regions that process RPEs, such as the dorsal anterior cingulate cortex (dACC), have been identified, only indirect evidence links timing and network organization of RPE processing in humans. In electroencephalography (EEG), which is well known for its high temporal resolution, the feedback-related negativity (FRN) has been suggested to reflect RPE processing. Recent studies, however, suggested that the FRN might reflect surprise, which would correspond to the absolute, rather than the signed RPE signals. Furthermore, the localization of the FRN remains a matter of debate. In this simultaneous EEG-functional magnetic resonance imaging (fMRI) study, we localized the FRN directly using the superior spatial resolution of fMRI without relying on any spatial constraint or other assumption. Using two different single-trial approaches, we consistently found a cluster within the dACC. One analysis revealed additional activations of the salience network. Furthermore, we evaluated the effect of signed RPEs and surprise signals on the FRN amplitude. We considered that both signals are usually correlated and found that only surprise signals modulate the FRN amplitude. Last, we explored the pathway of RPE signals using dynamic causal modeling (DCM). We found that the surprise signals are directly projected to the source region of the FRN. This finding contradicts earlier theories about the network organization of the FRN, but is in line with a recent theory stating that dopamine neurons also encode surprise-like saliency signals. Our findings crucially advance the understanding of the FRN. We found compelling evidence that the FRN originates from the dACC. Furthermore, we clarified the functional role of the FRN, and determined the role of the dACC within the RPE network. These findings should enable us to study the processing of surprise and adjustment signals in the dACC in healthy and also in psychiatric patients.

Adolescence is associated with quickly changing environmental demands which require excellent adaptive skills and high cognitive flexibility. Feedback-guided adaptive learning and cognitive flexibility are driven by reward prediction error (RPE) signals, which indicate the accuracy of expectations and can be estimated using computational models. Despite the importance of cognitive flexibility during adolescence, only little is known about how RPE processing in cognitive flexibility deviates between adolescence and adulthood.In this study, we investigated the developmental aspects of cognitive flexibility by means of computational models and functional magnetic resonance imaging (fMRI). We compared the neural and behavioral correlates of cognitive flexibility in healthy adolescents (12–16 years) to adults performing a probabilistic reversal learning task. Using a modified risk-sensitive reinforcement learning model, we found that adolescents learned faster from negative RPEs than adults. The fMRI analysis revealed that within the RPE network, the adolescents had a significantly altered RPE-response in the anterior insula. This effect seemed to be mainly driven by increased responses to negative prediction errors.In summary, our findings indicate that decision making in adolescence goes beyond merely increased reward-seeking behavior and provides a developmental perspective to the behavioral and neural mechanisms underlying cognitive flexibility in the context of reinforcement learning.

IMPORTANCE Attention-deficit/hyperactivity disorder (ADHD) has been associated with deficient decision making and learning. Models of ADHD have suggested that these deficits could be caused by impaired reward prediction errors (RPEs). Reward prediction errors are signals that indicate violations of expectations and are known to be encoded by the dopaminergic system. However, the precise learning and decision-making deficits and their neurobiological correlates in ADHD are not well known. OBJECTIVE To determine the impaired decision-making and learning mechanisms in juvenile ADHD using advanced computational models, as well as the related neural RPE processes using multimodal neuroimaging. DESIGN, SETTING, AND PARTICIPANTS Twenty adolescents with ADHD and 20 healthy adolescents serving as controls (aged 12-16 years) were examined using a probabilistic reversal learning task while simultaneous functional magnetic resonance imaging and electroencephalogram were recorded. MAIN OUTCOMES AND MEASURES Learning and decision making were investigated by contrasting a hierarchical Bayesian model with an advanced reinforcement learning model and by comparing the model parameters. The neural correlates of RPEs were studied in functional magnetic resonance imaging and electroencephalogram. RESULTS Adolescents with ADHD showed more simplistic learning as reflected by the reinforcement learning model (exceedance probability, P x = .92) and had increased exploratory behavior compared with healthy controls (mean [SD] decision steepness parameter β: ADHD, 4.83 [2.97]; controls, 6.04 [2.53]; P = .02). The functional magnetic resonance imaging analysis revealed impaired RPE processing in the medial prefrontal cortex during cue as well as during outcome presentation (P < .05, family-wise error correction). The outcome-related impairment in the medial prefrontal cortex could be attributed to deficient processing at 200 to 400 milliseconds after feedback presentation as reflected by reduced feedback-related negativity (ADHD, 0.61 [3.90] μV; controls, −1.68 [2.52] μV; P = .04). CONCLUSIONS AND RELEVANCE The combination of computational modeling of behavior and multimodal neuroimaging revealed that impaired decision making and learning mechanisms in adolescents with ADHD are driven by impaired RPE processing in the medial prefrontal cortex. This novel, combined approach furthers the understanding of the pathomechanisms in ADHD and may advance treatment strategies.

No abstract

Patients with obsessive-compulsive disorder (OCD) can be described as cautious and hesitant, manifesting an excessive indecisiveness that hinders efficient decision making. However, excess caution in decision making may also lead to better performance in specific situations where the cost of extended deliberation is small. We compared 16 juvenile OCD patients with 16 matched healthy controls whilst they performed a sequential information gathering task under different external cost conditions. We found that patients with OCD outperformed healthy controls, winning significantly more points. The groups also differed in the number of draws required prior to committing to a decision, but not in decision accuracy. A novel Bayesian computational model revealed that subjective sampling costs arose as a non-linear function of sampling, closely resembling an escalating urgency signal. Group difference in performance was best explained by a later emergence of these subjective costs in the OCD group, also evident in an increased decision threshold. Our findings present a novel computational model and suggest that enhanced information gathering in OCD can be accounted for by a higher decision threshold arising out of an altered perception of costs that, in some specific contexts, may be advantageous.