Temporal dynamics of prediction error processing during reward-based decision making

Philiastides, Marios G.; Biele, Guido; Vavatzanidis, Niki; Kazzer, Philipp; Heekeren, Hauke R.

doi:10.1016/j.neuroimage.2010.05.052

Cited by 107 publications

(114 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Berns and Bell (2012) point out that even when the two dimensions of outcome feedback are presented simultaneously, participants still have to read them sequentially. Therefore, feedback processing in a task which present valence and magnitude simultaneously also contains an initial encoding stage and a follow‐up integration stage (Philiastides, Biele, Vavatzanidis, Kazzer, & Heekeren, 2010). …”

Section: Discussionmentioning

confidence: 99%

“…We suggest that these results support our idea that the FRN amplitude is more sensitive to feedback ambiguity in the numerical context than in the symbolic indicator context; consequently, the effect of ambiguity was more prominent when the FRN was elicited by magnitude presentation than by valence presentation (second feedback phase). In line with this viewpoint, previous research has shown that compared with the P3, the FRN is more likely to reflect the characteristics of the information presented at the moment (Gu et al., 2011; Philiastides et al., 2010). …”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Valence and magnitude ambiguity in feedback processing

Xue

Broster

et al. 2017

Brain and Behavior

View full text Add to dashboard Cite

BackgroundOutcome feedback which indicates behavioral consequences are crucial for reinforcement learning and environmental adaptation. Nevertheless, outcome information in daily life is often totally or partially ambiguous. Studying how people interpret this kind of information would provide important knowledge about the human evaluative system.MethodsThis study concentrates on the neural processing of partially ambiguous feedback, that is, either its valence or magnitude is unknown to participants. To address this topic, we sequentially presented valence and magnitude information; electroencephalography (EEG) response to each kind of presentation was recorded and analyzed. The event‐related potential components feedback‐related negativity (FRN) and P3 were used as indices of neural activity.ResultsConsistent with previous literature, the FRN elicited by ambiguous valence was not significantly different from that elicited by negative valence. On the other hand, the FRN elicited by ambiguous magnitude was larger than both the large and small magnitude, indicating the motivation to seek unambiguous magnitude information. The P3 elicited by ambiguous valence and ambiguous magnitude was not significantly different from that elicited by negative valence and small magnitude, respectively, indicating the emotional significance of feedback ambiguity. Finally, the aforementioned effects also manifested in the stage of information integration.ConclusionThese findings indicate both similarities and discrepancies between the processing of valence ambiguity and that of magnitude ambiguity, which may help understand the mechanisms of ambiguous information processing.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Valence and magnitude ambiguity in feedback processing

Xue

Broster

et al. 2017

Brain and Behavior

View full text Add to dashboard Cite

show abstract

“…(Chase, Swainson, Durham, Benham, & Cools, 2011;Cohen & Ranganath, 2007;Philiastides, Biele, Vavatzanidis, Kazzer, & Heekeren, 2010). Other evidence for a model-free basis for the FRN comes from the demonstration that dopamine, the neurotransmitter implicated in generating the FRN, promotes model-free rather than model-based reinforcement learning (Wunderlich, Smittenaar, & Dolan, 2012).…”

Section: Modelmentioning

confidence: 99%

A neural reward prediction error revealed by a meta-analysis of ERPs using great grand averages.

Sambrook

Goslin

2015

Psychological Bulletin

329

388

View full text Add to dashboard Cite

Economic approaches to decision making assume that people attach values to prospective goods and act in order to maximise their obtained value. Neuroeconomics strives to observe these values directly in the brain. A widely used valuation term in formal learning and decision-making models is the reward prediction error: the value of an outcome relative to its expected value. An influential theory (Holroyd and Coles, 2002) claims that an electrophysiological component, the feedback related negativity (FRN), codes a reward prediction error in the human brain. Such a component should be sensitive to both the prior likelihood of reward and its magnitude on receipt. A number of studies have found the FRN to be insensitive to reward magnitude, thus questioning the Holroyd and Coles account.However, because of marked inconsistencies in how the FRN is measured, a meaningful synthesis of this evidence is highly problematic. We conducted a meta-analysis of the FRN's response to both reward magnitude and likelihood using a novel method in which published effect sizes were disregarded in favour of direct measurement of the published waveforms themselves, with these waveforms then averaged to produce "great-grand averages". Under this standardised measure, the meta-analysis revealed strong effects of magnitude and likelihood on the FRN consistent with it encoding a reward prediction error. In addition, it revealed strong main effects of reward magnitude and likelihood across much of the waveform, indicating sensitivity to unsigned prediction errors or "salience". The great grand average technique is proposed as a general method for meta-analysis of ERPs.Keywords: Feedback related negativity (FRN); Event-related potential (ERP); Reward prediction error (RPE); Unsigned prediction error; Meta-analysis; Great grand average Masked Manuscript without Author Information 2 Explaining human behavior under choice requires understanding how humans assign value to goods and actions. This valuation occurs at a nexus of psychological influences running from high level processes such as framing effects and counterfactual comparisons down to basic physiological influences such as satiation. It is likely to be dependent on an individual's knowledge both through conscious extrapolation from experience and simple reinforcement learning.Early attempts to explain human valuation were aimed at demonstrating that choice was entirely rational, and embodied key axioms of neoclassical economics such as expected utility. This approach employed a black box methodology, observing the "revealed preferences" of outward behavior in favor of the underlying apparatus of valuation, and treating humans only "as if" they computed utilities (Friedman, 1953;Samuelson, 1937).These assumptions have come under attack from the field of behavioral economics, which has succeeded in documenting widespread and consistent deviations from rational choice. A fully psychological approach, behavioral economics has endeavored to open the black box and consider a more va...

show abstract

“…The FRN amplitude is larger for negative feedback than for positive feedback, indicating that the FRN is sensitive to feedback valence (Gehring & Willoughby, 2002;Holroyd & Coles, 2002;Luu, Tucker, Derryberry, Reed, & Poulsen, 2003;Miltner, Braun, & Coles, 1997;Yeung, Holroyd, & Cohen, 2005). A current theory of the FRN assumes that it reflects a reinforcement signal that triggers learning from negative feedback (Holroyd & Coles, 2002), which explains why FRN amplitude (Bellebaum & Daum, 2008;Cohen & Ranganath, 2007;Philiastides, Biele, Vavatzanidis, Kazzer, & Heekeren, 2010;van der Helden, Boksem, & Blom, 2010) or oscillatory activity related to the FRN (Cavanagh, Frank, Klein, & Allen, 2010) predicts the strength of behavioral adjustments in simple decision tasks.…”

Section: Erps Related To Feedback Processingmentioning

confidence: 99%

Feedback-related brain activity predicts learning from feedback in multiple-choice testing

Ernst

Steinhauser

2012

Cogn Affect Behav Neurosci

View full text Add to dashboard Cite

Different event-related potentials (ERPs) have been shown to correlate with learning from feedback in decisionmaking tasks and with learning in explicit memory tasks. In the present study, we investigated which ERPs predict learning from corrective feedback in a multiple-choice test, which combines elements from both paradigms. Participants worked through sets of multiple-choice items of a Swahili-German vocabulary task. Whereas the initial presentation of an item required the participants to guess the answer, corrective feedback could be used to learn the correct response. Initial analyses revealed that corrective feedback elicited components related to reinforcement learning (FRN), as well as to explicit memory processing (P300) and attention (early frontal positivity). However, only the P300 and early frontal positivity were positively correlated with successful learning from corrective feedback, whereas the FRN was even larger when learning failed. These results suggest that learning from corrective feedback crucially relies on explicit memory processing and attentional orienting to corrective feedback, rather than on reinforcement learning.

show abstract

Temporal dynamics of prediction error processing during reward-based decision making

Cited by 107 publications

References 56 publications

Valence and magnitude ambiguity in feedback processing

Valence and magnitude ambiguity in feedback processing

A neural reward prediction error revealed by a meta-analysis of ERPs using great grand averages.

Feedback-related brain activity predicts learning from feedback in multiple-choice testing

Contact Info

Product

Resources

About