Computational reinforcement learning, reward (and punishment), and dopamine in psychiatric disorders

Abstract

In the DSM-5, psychiatric diagnoses are made based on self-reported symptoms and clinician-identified signs. Though helpful in choosing potential interventions based on the available regimens, this conceptualization of psychiatric diseases can limit basic science investigation into their underlying causes. The reward prediction error (RPE) hypothesis of dopamine neuron function posits that phasic dopamine signals encode the difference between the rewards a person expects and experiences. The computational framework from which this hypothesis was derived, temporal difference reinforcement learning (TDRL), is largely focused on reward processing rather than punishment learning. Many psychiatric disorders are characterized by aberrant behaviors, expectations, reward processing, and hypothesized dopaminergic signaling, but also characterized by suffering and the inability to change one's behavior despite negative consequences. In this review, we provide an overview of the RPE theory of phasic dopamine neuron activity and review the gains that have been made through the use of computational reinforcement learning theory as a framework for understanding changes in reward processing. The relative dearth of explicit accounts of punishment learning in computational reinforcement learning theory and its application in neuroscience is highlighted as a significant gap in current computational psychiatric research. Four disorders comprise the main focus of this review: two disorders of traditionally hypothesized hyperdopaminergic function, addiction and schizophrenia, followed by two disorders of traditionally hypothesized hypodopaminergic function, depression and post-traumatic stress disorder (PTSD). Insights gained from a reward processing based reinforcement learning framework about underlying dopaminergic mechanisms and the role of punishment learning (when available) are explored in each disorder. Concluding remarks focus on the future directions required to characterize neuropsychiatric disorders with a hypothesized cause of underlying dopaminergic transmission.

Combining noninvasive brain stimulation with behavioral pharmacology methods to study mechanisms of substance use disorder

“…Previous fMRI studies, including our own, have demonstrated that the left DLPFC is involved in decision-making, working memory, and attention processes ( Wesley et al, 2011 , 2014 ; Wesley and Bickel, 2014 ). Fronto-striatal circuits that involve the DLPFC have also been implicated in reinforcement-based computational models of learning and memory ( Lipton et al, 2019 ; Volkow et al, 2019 ; Averbeck and O'Doherty, 2022 ; Liebenow et al, 2022 ). Consistent with a role in CUD, left DLPFC function predicted cannabis versus money choice ( Bedi et al, 2015 ).…”