Responses of human frontal cortex to surprising events are predicted by formal associative learning theory

Fletcher, Paul C.; Anderson, John M.; Shanks, David R.; Honey, R.A.E.; Carpenter, T. Adrian; Donovan, Tim; Papadakis, N.; Bullmore, Edward T.

doi:10.1038/nn733

Cited by 203 publications

(192 citation statements)

References 28 publications

Supporting

Mentioning

165

Contrasting

Order By: Relevance

“…The finding is consistent with other recently published studies (40,41) as well as with dopaminergic inputs to this region of the brain (42). Evaluation of expected reward level (equivalently, risk of the choice) may thus take place within this region of the brain, or may be carried out through interaction with other reward͞emotion-related structures to which it is connected (such as amygdala, basal ganglia, insula, and other sectors of PFC).…”

Section: Discussionsupporting

confidence: 93%

Electrophysiological correlates of reward prediction error recorded in the human prefrontal cortex

Oya

Adolphs

Kawasaki

et al. 2005

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Lesion and functional imaging studies have shown that the ventromedial prefrontal cortex is critically involved in the avoidance of risky choices. However, detailed descriptions of the mechanisms that underlie the establishment of such behaviors remain elusive, due in part to the spatial and temporal limitations of available research techniques. We investigated this issue by recording directly from prefrontal depth electrodes in a rare neurosurgical patient while he performed the Iowa Gambling Task, and we concurrently measured behavioral, autonomic, and electrophysiological responses. We found a robust alpha-band component of event-related potentials that reflected the mismatch between expected outcomes and actual outcomes in the task, correlating closely with the reward-related error obtained from a reinforcement learning model of the patient's choice behavior. The finding implicates this brain region in the acquisition of choice bias by means of a continuous updating of expectations about reward and punishment.decision-making ͉ emotion

show abstract

Section: Discussionsupporting

confidence: 93%

Electrophysiological correlates of reward prediction error recorded in the human prefrontal cortex

Oya

Adolphs

Kawasaki

et al. 2005

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

show abstract

“…Small action prediction errors allow the learning participant to reconfirm that they are engaging in the correct strategy without observing the outcome of the other player. The region of DLPFC where activity correlates most strongly with the expected action prediction error signal has previously been shown to respond to other types of prediction error (15), to conflict (16), and to trials that violate an expectancy that was learned from previous trials (17). The present finding also fits well with previous research implicating the DLPFC in action selection (18).…”

Section: Discussionsupporting

confidence: 90%

Neural mechanisms of observational learning

Burke

Tobler

Baddeley

et al. 2010

Proc. Natl. Acad. Sci. U.S.A.

289

343

View full text Add to dashboard Cite

Individuals can learn by interacting with the environment and experiencing a difference between predicted and obtained outcomes (prediction error). However, many species also learn by observing the actions and outcomes of others. In contrast to individual learning, observational learning cannot be based on directly experienced outcome prediction errors. Accordingly, the behavioral and neural mechanisms of learning through observation remain elusive. Here we propose that human observational learning can be explained by two previously uncharacterized forms of prediction error, observational action prediction errors (the actual minus the predicted choice of others) and observational outcome prediction errors (the actual minus predicted outcome received by others). In a functional MRI experiment, we found that brain activity in the dorsolateral prefrontal cortex and the ventromedial prefrontal cortex respectively corresponded to these two distinct observational learning signals.prediction error | reward | vicarious learning | dorsolateral prefrontal cortex | ventromedial prefrontal cortex I n uncertain and changing environments, flexible control of actions has individual and evolutionary advantages by allowing goaldirected and adaptive behavior. Flexible action control requires an understanding of how actions bring about rewarding or punishing outcomes. Through instrumental conditioning, individuals can use previous outcomes to modify future actions (1-4). However, individuals learn not only from their own actions and outcomes but also from those that are observed. One of the most illustrative examples of observational learning happens in Antarctica, where flocks of Adelie penguins often congregate at the water's edge to enter the sea and feed on krill. However, the main predator of the penguins, the leopard seal, is often lurking out of sight beneath the waves, making it a risky prospect to be the first one to take the plunge. As this waiting game develops, one of the animals often becomes so hungry that it jumps, and if no seal appears the rest of the group will all follow suit. The following penguins make a decision after observing the action and outcome of the first (5). This ability to learn from observed actions and outcomes is a pervasive feature of many species and can be absolutely crucial when the stakes are high. For example, predator avoidance techniques or the eating of a novel food item are better learned from another's experience rather than putting oneself at risk with trial-and-error learning. Although we know a fair amount about the neural mechanisms of individuals learning about their own actions and outcomes (6), almost nothing is known about the brain processes involved when individuals learn from observed actions and outcomes (7). This lack of knowledge is all the more surprising given that observational learning is such a wide-ranging phenomenon.In this study, 21 participants engaged in a novel observational learning task based on a simple two-armed bandit problem (Fig. 1A) while being scanne...

show abstract

“…It is tempting to interpret these responses as relevant for the new learning that has to occur in this phase: while CSX (as part of CSAX) has to be established as nonpredictive for the US, CSY (as part of CSBY) has to be established as predictive for the US. Previous studies would support such an interpretation, linking dlPFC responses to accurate contingency tracking (Carter et al, 2006) and to unsigned prediction error signals as observed in causal learning studies (Fletcher et al, 2001;Turner et al, 2004). Yet, confounding factors such as stimulus novelty and stimulus complexity were also present on compound trials and could thus account for the increase in dlPFC activity (Turner et al, 2004).…”

Section: Discussionmentioning

confidence: 72%

Neurobiological Mechanisms Underlying the Blocking Effect in Aversive Learning

2012

View full text Add to dashboard Cite

Current theories of classical conditioning assume that learning depends on the predictive relationship between events, not just on their temporal contiguity. Here we employ the classic experiment substantiating this reasoning-the blocking paradigm-in combination with functional magnetic resonance imaging (fMRI) to investigate whether human amygdala responses in aversive learning conform to these assumptions. In accordance with blocking, we demonstrate that significantly stronger behavioral and amygdala responses are evoked by conditioned stimuli that are predictive of the unconditioned stimulus than by conditioned stimuli that have received the same pairing with the unconditioned stimulus, yet have no predictive value. When studying the development of this effect, we not only observed that it was related to the strength of previous conditioned responses, but also that predictive compared with nonpredictive conditioned stimuli received more overt attention, as measured by fMRI-concurrent eye tracking, and that this went along with enhanced amygdala responses. We furthermore observed that prefrontal regions play a role in the development of the blocking effect: ventromedial prefrontal cortex (subgenual anterior cingulate) only exhibited responses when conditioned stimuli had to be established as nonpredictive for an outcome, whereas dorsolateral prefrontal cortex also showed responses when conditioned stimuli had to be established as predictive. Most importantly, dorsolateral prefrontal cortex connectivity to amygdala flexibly switched between positive and negative coupling, depending on the requirements posed by predictive relationships. Together, our findings highlight the role of predictive value in explaining amygdala responses and identify mechanisms that shape these responses in human fear conditioning.

show abstract

Responses of human frontal cortex to surprising events are predicted by formal associative learning theory

Cited by 203 publications

References 28 publications

Electrophysiological correlates of reward prediction error recorded in the human prefrontal cortex

Electrophysiological correlates of reward prediction error recorded in the human prefrontal cortex

Neural mechanisms of observational learning

Neurobiological Mechanisms Underlying the Blocking Effect in Aversive Learning

Contact Info

Product

Resources

About