Effects of feedback reliability on feedback-related brain activity: A feedback valuation account

Abstract

Adaptive decision making relies on learning from feedback. Because feedback sometimes can be misleading, optimal learning requires that knowledge about the feedback's reliability be utilized to adjust feedback processing. Although previous research has shown that feedback reliability indeed influences feedback processing, the underlying mechanisms through which this is accomplished remain unclear. Here we propose that feedback processing is adjusted by the adaptive, top-down valuation of feedback. We assume that unreliable feedback is devalued relative to reliable feedback, thus reducing the reward prediction errors that underlie feedback-related brain activity and learning. A crucial prediction of this account is that the effects of feedback reliability are susceptible to contrast effects. That is, the effects of feedback reliability should be enhanced when both reliable and unreliable feedback are experienced within the same context, as compared to when only one level of feedback reliability is experienced. To evaluate this prediction, we measured the event-related potentials elicited by feedback in two experiments in which feedback reliability was varied either within or between blocks. We found that the fronto-central valence effect, a correlate of reward prediction errors during reinforcement learning, was reduced for unreliable feedback. But this result was obtained only when feedback reliability was varied within blocks, thus indicating a contrast effect. This suggests that the adaptive valuation of feedback is one mechanism underlying the effects of feedback reliability on feedback processing.

Differential effects of instructed and objective feedback reliability on feedback‐related brain activity

“…Because unreliable feedback is misleading and thus detrimental for decision making, it is essential to consider not only feedback itself but also its reliability when evaluating a decision. While several recent studies could show that unreliable feedback is accompanied by reduced feedback‐related brain activity (Ernst & Steinhauser, , ; Li, Peng, Li, & Holroyd, ; Schiffer, Siletti, Waszak, & Yeung, ; Walentowska, Moors, Paul, & Pourtois, ), it still unclear whether these effects reflect top‐down control over feedback processing due to knowledge about feedback reliability or whether they are simply due to learning a decreased feedback‐outcome contingency. To address this question, we considered feedback‐related brain activity in ERPs in two experiments in which two levels of feedback reliability were instructed (Experiment 1) or objectively manipulated (Experiment 2).…”

“…Several studies investigated effects of feedback reliability on these ERP components using a variety of paradigms, including reversal learning (Schiffer et al, ), time estimation (Li, Peng, et al, ), a go/no‐go task (Walentowska et al, ), or a learn‐test paradigm (Ernst & Steinhauser, , ). Common to all these tasks is that participants have to learn the correct response (Ernst & Steinhauser, , ; Schiffer et al, ) or optimize their decision behavior (Li, Peng, et al, ), while being informed that the feedback is either unreliable, meaning that it often does not reflect their true performance or is uninformative. In most studies, instructions were provided that truthfully informed about the actual feedback reliability, sometimes additionally cued by a stimulus (Li, Peng, et al, ) or stimulus characteristics (Ernst & Steinhauser, ; Walentowska et al, ).…”

“…Common to all these tasks is that participants have to learn the correct response (Ernst & Steinhauser, , ; Schiffer et al, ) or optimize their decision behavior (Li, Peng, et al, ), while being informed that the feedback is either unreliable, meaning that it often does not reflect their true performance or is uninformative. In most studies, instructions were provided that truthfully informed about the actual feedback reliability, sometimes additionally cued by a stimulus (Li, Peng, et al, ) or stimulus characteristics (Ernst & Steinhauser, ; Walentowska et al, ). Some of these studies reported a reduced fronto‐central valence effect for unreliable compared to reliable feedback (Ernst & Steinhauser, , ; Schiffer et al, ; Walentowska et al, ) suggesting that feedback reliability influences the generation or utilization of reward prediction errors, while others did not find such an effect (Li, Peng, et al, ; Walentowska et al, , experiment 2).…”

“…In most studies, instructions were provided that truthfully informed about the actual feedback reliability, sometimes additionally cued by a stimulus (Li, Peng, et al, ) or stimulus characteristics (Ernst & Steinhauser, ; Walentowska et al, ). Some of these studies reported a reduced fronto‐central valence effect for unreliable compared to reliable feedback (Ernst & Steinhauser, , ; Schiffer et al, ; Walentowska et al, ) suggesting that feedback reliability influences the generation or utilization of reward prediction errors, while others did not find such an effect (Li, Peng, et al, ; Walentowska et al, , experiment 2). Moreover, most of these studies also reported a reduced P3 following unreliable feedback compared to reliable feedback (Ernst & Steinhauser, ; Schiffer et al, ; Walentowska et al, ), suggesting that feedback reliability also influences feedback processing by an explicit learning mechanism.…”

“…Based on the effects of feedback reliability on the fronto‐central valence effect, we have proposed that unreliable feedback implies a suppression of the generation or utilization of reward prediction errors via top‐down control (Ernst & Steinhauser, ) or a top‐down devaluation of unreliable feedback (Ernst & Steinhauser, ). In general, concepts of top‐down control over RL are in line with studies showing that false instructions about reward contingencies can bias RL in a trial‐and‐error learning paradigm (Doll, Jacobs, Sanfey, & Frank, ), which is particularly pronounced in participants with a more stable representation of the misleading instruction (Doll, Hutchison, & Frank, ).…”