An Automatic Valuation System in the Human Brain: Evidence from Functional Neuroimaging

328

386

Uncertainty about the function of orbitofrontal cortex (OFC) in guiding decision-making may be a result of its medial (mOFC) and lateral (lOFC) divisions having distinct functions. Here we test the hypothesis that the mOFC is more concerned with reward-guided decision making, in contrast with the lOFC's role in reward-guided learning. Macaques performed three-armed bandit tasks and the effects of selective mOFC lesions were contrasted against lOFC lesions. First, we present analyses that make it possible to measure reward-credit assignment-a crucial component of reward-value learning-independently of the decisions animals make. The mOFC lesions do not lead to impairments in reward-credit assignment that are seen after lOFC lesions. Second, we examined how the reward values of choice options were compared. We present three analyses, one of which examines reward-guided decision making independently of reward-value learning. Lesions of the mOFC, but not the lOFC, disrupted reward-guided decision making. Impairments after mOFC lesions were a function of the multiple option contexts in which decisions were made. Contrary to axiomatic assumptions of decision theory, the mOFC-lesioned animals' value comparisons were no longer independent of irrelevant alternatives.A lthough it is widely agreed that the orbitofrontal cortex (OFC) is important for value-guided behavior (1, 2), the precise nature of its contribution is unclear. On the one hand, neuroimaging experiments suggest its ventromedial aspect is part of the circuit for deciding between options on the basis of their value (3, 4). Moreover De Martino et al. (5) have reported that ventromedial frontal activity is correlated with the degree to which subjects' decisions are rational and uninfluenced by irrelevant features of the context in which they are made. In contrast, other accounts emphasize a role in learning and storage of option values (6).One potential reason for lack of consensus about the OFC may be that there is functional specialization within it, as suggested by the distinct anatomical connections of its ventromedial and lateral divisions (7) The aim of our study was, therefore, to assess evidence of functional specialization within the OFC for value learning and value-guided decision making. The effect of medial OFC (mOFC) lesions was compared with that of lateral OFC (lOFC) lesions in two groups of animals.The investigation of value-guided decision making was based on the premise that a lesion that disrupts part a circuit for deciding and discriminating between choice options of differing value should impair decision making as a function of the proximity of options' values. By analogy, lesions of visual feature-discrimination mechanisms disrupt discrimination as a function of the similarity of the visual features (8). In experiments reported below, macaques were trained to choose between three stimuli associated with differing probabilities of reward. The probabilities of reward, therefore, determined the options' values and the proximity of the best (...

Section: Discussionsupporting

confidence: 44%

“…S6). Medial OFC value signals, even if automatically generated (24), instead appear to be critical for guiding decisions.…”

Section: Discussionmentioning

confidence: 99%

Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex

Noonan

Walton

Behrens

et al. 2010

328

386

“…Positive correlation between activity and the values of chosen options and negative correlations with values of rejected options suggests a role in decision making (21) or attentional selection (23). However, there is also evidence the vmPFC tracks values of items even in the absence of any decision or when watching somebody else choosing (24,25). Some studies have suggested that these value representations are independent of reward type (money, food) (26) and reflect the impact of other Fig.…”

Section: Resultsmentioning

confidence: 99%

Connectivity reveals relationship of brain areas for reward-guided learning and decision making in human and monkey frontal cortex

Neubert

Mars

Sallet

et al. 2015

350

388

Reward-guided decision-making depends on a network of brain regions. Among these are the orbitofrontal and the anterior cingulate cortex. However, it is difficult to ascertain if these areas constitute anatomical and functional unities, and how these areas correspond between monkeys and humans. To address these questions we looked at connectivity profiles of these areas using resting-state functional MRI in 38 humans and 25 macaque monkeys. We sought brain regions in the macaque that resembled 10 human areas identified with decision making and brain regions in the human that resembled six macaque areas identified with decision making. We also used diffusion-weighted MRI to delineate key human orbital and medial frontal brain regions. We identified 21 different regions, many of which could be linked to particular aspects of reward-guided learning, valuation, and decision making, and in many cases we identified areas in the macaque with similar coupling profiles.orbitofrontal cortex | anterior cingulate cortex | decision making | resting state functional connectivity | comparative anatomy A s humans we make decisions by taking into account different types of information, weighing our options carefully, and eventually coming to a conclusion. We then learn from witnessing the outcome of our decisions. Human functional MRI (fMRI) has had a major impact on elucidating the neural networks mediating decision making and learning, but key insights can only be obtained in neural recording, stimulation, and focal lesion studies conducted in animal models, such as the macaque. Combining insights from human fMRI and animal studies is, however, not straightforward because there is uncertainty about basic issues, such as anatomical and functional correspondences between species (1). For example, although there are many reports of decision value-related activity in the human ventromedial prefrontal cortex (vmPFC) (2, 3), it is unclear whether they can be related to reports of reward-related activity either on the ventromedial surface of the frontal lobe (4, 5), in the adjacent medial orbitofrontal sulcus (6), or indeed to any macaque brain area. It is claimed that some areas implicated in rewardguided decision making and learning, such as parts of anterior cingulate cortex (ACC), are not found in macaques (7), but such theories have never been formally tested.In addition, there is uncertainty about the basic constituent components of decision-making and learning circuits. To return to the example of the vmPFC, although this region is often contrasted with similarly large subdivisions of the frontal cortex, such as the lateral orbitofrontal cortex (lOFC) and ACC (8), it is unclear whether, and if so how, it should be decomposed into further subdivisions. Moreover, there are sometimes fundamental disagreements about how brain areas contribute to decision making and learning. For example, it has been claimed both that the ACC does (9-11) and does not (12) contribute to reward-based decision making and that it is concerned with dist...

“…Unlike our current understanding of the valuation process for self-regarding choices (3,(6)(7)(8)(9)(10)(11)(12), it is much less clear whether learning also can be driven by other-regarding values, and whether this other-regarding valuation relies on the same mechanisms of reinforcement learning as those used for self. Moreover, despite the rapidly accumulating research on reward processing in social domains (13)(14)(15)(16)(17)(18)(19), the question remains of how neural representation of self-regarding vs. other-regarding values is related to individual differences in altruistic behavior.…”

mentioning

confidence: 87%

Spatial gradient in value representation along the medial prefrontal cortex reflects individual differences in prosociality

Sul

Tobler

Hein

et al. 2015

123

112

Despite the importance of valuing another person's welfare for prosocial behavior, currently we have only a limited understanding of how these values are represented in the brain and, more importantly, how they give rise to individual variability in prosociality. In the present study, participants underwent functional magnetic resonance imaging while performing a prosocial learning task in which they could choose to benefit themselves and/or another person. Choice behavior indicated that participants valued the welfare of another person, although less so than they valued their own welfare. Neural data revealed a spatial gradient in activity within the medial prefrontal cortex (MPFC), such that ventral parts predominantly represented self-regarding values and dorsal parts predominantly represented other-regarding values. Importantly, compared with selfish individuals, prosocial individuals showed a more gradual transition from selfregarding to other-regarding value signals in the MPFC and stronger MPFC-striatum coupling when they made choices for another person rather than for themselves. The present study provides evidence of neural markers reflecting individual differences in human prosociality.R anging from small acts of kindness in daily life to self-sacrificing altruism under life-threatening situations, we often observe large individual differences in how humans value another person's welfare. This differential valuation process seems to be the key to understanding various human prosocial behaviors, which are fundamental to the sustainability of human society (1). The underlying neural mechanisms and their relationship to individual differences in prosociality remain unclear, however.Perhaps the most powerful way of assessing how an outcome is valued is to use an instrumental learning paradigm that examines whether the occurrence of a response increases when it is followed by that outcome (2). The mechanisms underlying this type of learning have been described more formally with a computational model, known as the advantage learning model (3-5), which has been used successfully to reveal the neuroanatomical substrates of subjective valuation (3,4,6). Previous research has further refined the neurobiological model of reinforcement learning by emphasizing the specific roles played by the medial frontal cortex and the striatum; the medial frontal cortex computes the value of the chosen action, whereas the striatum processes reward prediction errors during reinforcement learning (4, 6-10).Unlike our current understanding of the valuation process for self-regarding choices (3, 6-12), it is much less clear whether learning also can be driven by other-regarding values, and whether this other-regarding valuation relies on the same mechanisms of reinforcement learning as those used for self. Moreover, despite the rapidly accumulating research on reward processing in social domains (13-19), the question remains of how neural representation of self-regarding vs. other-regarding values is related to individual differen...