Model‐Based fMRI and Its Application to Reward Learning and Decision Making

O’Doherty, John P.; Hampton, Alan N.; Kim, Hackjin

doi:10.1196/annals.1390.022

Cited by 461 publications

(393 citation statements)

References 28 publications

Supporting

Mentioning

376

Contrasting

Unclassified

Order By: Relevance

“…This approach is a unique contribution to the expanding field of computational model-based fMRI methods (63). In particular, the model-based RSA method we propose is a fundamental departure from the typical model-based fMRI approach (21,64,65) that focuses on localizing time-varying model parameters to activation timeseries of brain regions.…”

Section: Discussionmentioning

confidence: 99%

Dynamic updating of hippocampal object representations reflects new conceptual knowledge

Mack

Love

Preston

2016

Proc. Natl. Acad. Sci. U.S.A.

206

248

View full text Add to dashboard Cite

Concepts organize the relationship among individual stimuli or events by highlighting shared features. Often, new goals require updating conceptual knowledge to reflect relationships based on different goal-relevant features. Here, our aim is to determine how hippocampal (HPC) object representations are organized and updated to reflect changing conceptual knowledge. Participants learned two classification tasks in which successful learning required attention to different stimulus features, thus providing a means to index how representations of individual stimuli are reorganized according to changing task goals. We used a computational learning model to capture how people attended to goal-relevant features and organized object representations based on those features during learning. Using representational similarity analyses of functional magnetic resonance imaging data, we demonstrate that neural representations in left anterior HPC correspond with model predictions of concept organization. Moreover, we show that during early learning, when concept updating is most consequential, HPC is functionally coupled with prefrontal regions. Based on these findings, we propose that when task goals change, object representations in HPC can be organized in new ways, resulting in updated concepts that highlight the features most critical to the new goal.category learning | attention | computational modeling | hippocampus | fMRI C oncepts are organizing principles that define how items or events are similar to one another. Goals are critical to shaping concepts, by emphasizing some shared features over others. When goals change, previously experienced events may be organized in new ways, resulting in an updated concept that highlights the features most critical to the new goal. For instance, consider purchasing a home. One must learn which features make for the most desirable home. A young couple seeking a cosmopolitan lifestyle may organize potential houses based on trendy features like exposed brick walls, a wet bar, and room for vintage record collections. However, with the news of a baby on the way, the couple's goals are likely to shift. After pouring through parenting books and web forums to learn what makes for a child-friendly home, they may look at those previously seen potential homes in a different light. Instead, family-oriented features such as whether or not a home has a bathtub, is within walking distance to a park, and is in a well-respected school district may matter more resulting in a reorganization of which homes are a good buy. At the core of this example are the fundamental challenges we face in flexible goaldirected learning. When learning new concepts (e.g., child-friendly instead of a trendy house), attention changes focus to different information and items that were conceptually dissimilar (e.g., two houses with and without a wet bar) may become more similar (e.g., they both are close to a park) and vice versa (1). Understanding how conceptual knowledge is created and updated during learning is a cen...

show abstract

Section: Discussionmentioning

confidence: 99%

Dynamic updating of hippocampal object representations reflects new conceptual knowledge

Mack

Love

Preston

2016

Proc. Natl. Acad. Sci. U.S.A.

206

248

View full text Add to dashboard Cite

show abstract

“…Contemporary functional magnetic resonance imaging (fMRI) studies of decision making in human volunteers involve parameterized models that interpret changes in blood-oxygenation-level-dependent (BOLD) signal in terms of (phasic and tonic) activity of the midbrain dopamine neurones and their limbic, striatal, and cortical projection structures (D'Ardenne et al, 2008;Knutson and Wimmer, 2007;O'Doherty et al, 2007).…”

Section: Scope Of Reviewmentioning

confidence: 99%

The Roles of Dopamine and Serotonin in Decision Making: Evidence from Pharmacological Experiments in Humans

Rogers

2010

Neuropsychopharmacol

195

130

View full text Add to dashboard Cite

Neurophysiological experiments in primates, alongside neuropsychological and functional magnetic resonance investigations in humans, have significantly enhanced our understanding of the neural architecture of decision making. In this review, I consider the more limited database of experiments that have investigated how dopamine and serotonin activity influences the choices of human adults. These include those experiments that have involved the administration of drugs to healthy controls, experiments that have tested genotypic influences upon dopamine and serotonin function, and, finally, some of those experiments that have examined the effects of drugs on the decision making of clinical samples. Pharmacological experiments in humans are few in number and face considerable methodological challenges in terms of drug specificity, uncertainties about pre-vs post-synaptic modes of action, and interactions with baseline cognitive performance. However, the available data are broadly consistent with current computational models of dopamine function in decision making and highlight the dissociable roles of dopamine receptor systems in the learning about outcomes that underpins value-based decision making. Moreover, genotypic influences on (interacting) prefrontal and striatal dopamine activity are associated with changes in choice behavior that might be relevant to understanding exploratory behaviors and vulnerability to addictive disorders. Manipulations of serotonin in laboratory tests of decision making in human participants have provided less consistent results, but the information gathered to date indicates a role for serotonin in learning about bad decision outcomes, non-normative aspects of risk-seeking behavior, and social choices involving affiliation and notions of fairness. Finally, I suggest that the role played by serotonin in the regulation of cognitive biases, and representation of context in learning, point toward a role in the cortically mediated cognitive appraisal of reinforcers when selecting between actions, potentially accounting for its influence upon the processing salient aversive outcomes and social choice.

show abstract

“…Instead of using the condition structure of the experiment to inform the analysis, as in a conventional fMRI analysis, model-based fMRI uses information derived from a computational model (12,13). For example, Daw et al (14) fitted a mathematical reinforcement learning model to the behavior of their study participants and then used parameter values of the model as regressors in the fMRI analysis.…”

mentioning

confidence: 99%

Using model-based functional MRI to locate working memory updates and declarative memory retrievals in the fronto-parietal network

Borst

Anderson

2013

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

In this study, we used model-based functional MRI (fMRI) to locate two functions of the fronto-parietal network: declarative memory retrievals and updating of working memory. Because regions in the fronto-parietal network are by definition coherently active, locating functions within this network is difficult. To overcome this problem, we applied model-based fMRI, an analysis method that uses predictions of a computational model to inform the analysis. We applied model-based fMRI to five previously published datasets with associated computational cognitive models, and subsequently integrated the results in a meta-analysis. The meta-analysis showed that declarative memory retrievals correlated with activity in the inferior frontal gyrus and the anterior cingulate, whereas updating of working memory corresponded to activation in the inferior parietal lobule, as well as to activation around the inferior frontal gyrus and the anterior cingulate.I n this study, we used model-based functional MRI (fMRI) to locate two functions of the so-called "fronto-parietal network." The fronto-parietal network consists of brain areas that are coherently active and assumed to implement cognitive control functions: working memory, attentional selection, and error monitoring (1-5). It typically involves at least the dorsolateral prefrontal cortex, the anterior cingulate, and a region around the intraparietal sulcus. Because the regions in the fronto-parietal network are by definition active at the same time, distinguishing the precise functional characteristics of those regions is difficult with conventional fMRI methods. Model-based fMRI is particularly well suited for dissociating between highly correlated contributions to the blood oxygen level-dependent (BOLD) signal. We used model-based fMRI to locate two functions within the fronto-parietal network: declarative memory retrieval and updating of working memory. These functions are often assumed to be part of the fronto-parietal network, but the hypothesized locations within the network differ among studies (2, 6-11).Model-based fMRI is a relatively recent approach to analyzing fMRI data. Instead of using the condition structure of the experiment to inform the analysis, as in a conventional fMRI analysis, model-based fMRI uses information derived from a computational model (12, 13). For example, Daw et al. (14) fitted a mathematical reinforcement learning model to the behavior of their study participants and then used parameter values of the model as regressors in the fMRI analysis. This resulted in brain regions that correlated significantly with those parameter values, and thus with certain features of their model. Another recent study showed that modelbased fMRI can be used successfully in combination with more high-level information-processing models (15). By regressing the activity of model components (e.g., visual processing, declarative memory retrieval) against neural activity, the neural correlates of the model components can be located.To identify the neural correlates o...

show abstract

Model‐Based fMRI and Its Application to Reward Learning and Decision Making

Cited by 461 publications

References 28 publications

Dynamic updating of hippocampal object representations reflects new conceptual knowledge

Dynamic updating of hippocampal object representations reflects new conceptual knowledge

The Roles of Dopamine and Serotonin in Decision Making: Evidence from Pharmacological Experiments in Humans

Using model-based functional MRI to locate working memory updates and declarative memory retrievals in the fronto-parietal network

Contact Info

Product

Resources

About