Monetary feedback modulates performance and electrophysiological indices of belief updating in reward learning

Abstract

Belief updating entails the incorporation of new information about the environment into internal models of the world. Bayesian inference is the statistically optimal strategy for performing belief updating in the presence of uncertainty. An important open question is whether the use of cognitive strategies that implement Bayesian inference is dependent upon motivational state and, if so, how this is reflected in electrophysiological signatures of belief updating in the brain. Here, we recorded the EEG of participants performing a simple reward learning task with both monetary and nonmonetary instructive feedback conditions. Our aim was to distinguish the influence of the rewarding properties of feedback on belief updating from the information content of the feedback itself. A Bayesian updating model allowed us to quantify different aspects of belief updating across trials, including the size of belief updates and the uncertainty of beliefs. Faster learning rates were observed in the monetary feedback condition compared to the instructive feedback condition, while belief updates were generally larger, and belief uncertainty smaller, with monetary compared to instructive feedback. Larger amplitudes in the monetary feedback condition were found for three ERP components: the P3a, the feedback‐related negativity, and the late positive potential. These findings suggest that motivational state influences inference strategies in reward learning, and this is reflected in the electrophysiological correlates of belief updating.

Between heuristics and optimality: Flexible integration of cost and evidence during information sampling

“…We examined the context of varying reward stakes on information sampling and found that while individuals maintained the same underlying strategy between both contexts, reward increased the overall information that people gathered. Our results differ from that observed in a reinforcement learning task, where reward stakes resulted in a switch to a more intensive but optimal strategy (Bennett et al, 2019). Additional investigation is needed into how individuals use context to evaluate when to switch information sampling strategies and when to adapt an existing strategy.…”

“…Behaviorally, participants in the high-reward condition over-sampled, effectively raising the stopping threshold without changing the underlying computations; both drawing more samples and waiting until they achieved a larger difference between categories. This stands in contrast with prior work that suggested increasing monetary stakes can increase one’s motivational state, thereby encouraging the use of more optimal but expensive strategies (Bennett et al, 2019; Manohar et al, 2015). Why might increased monetary stakes encourage behavioral adjustments but not push participants towards optimal behavior?…”

“…Lastly, we investigated whether an increase in reward stakes shifted strategy use across our participants. We hypothesized that under high reward contexts, participants might expend more computational resources to sample more optimally (Bennett et al, 2019; Manohar et al, 2015), thus participants’ behavior would be more similar to the Ideal Observer or be better fit by a strategy that relies on optimal value estimation (e.g., the EV-UT strategy). To test this, we split trials by reward condition within each participant and fit each subset of behavior to the ED-CA strategy and to the EV-UT strategy.…”

“…Indeed, evidence suggests that humans forgo using intensive updating computations, such as Bayesian inference (Charness & Levin, 2005; Gigerenzer & Goldstein, 1996; Payzan-LeNestour & Bossaerts, 2011; Steyvers et al, 2009), even for simpler decisions (Cassey et al, 2016). Thus, it is unclear whether strategies that rely on computations of expected value would guide information sampling decisions in humans, or whether the use of such optimal computations would depend on the reward value for a correct decision (Bennett et al, 2019; Manohar et al, 2015).…”

How predictability affects habituation to novelty

“…In recent years, some studies have investigated habituation to stimuli using P300 [24][25][26][27], but the effects of predictability (uncertainties and prediction errors) on brain activity related to novelty habituation remain unclear. Kopp (2007) was the first to study the relationship between Bayesian inference and P300 components [28], and several empirical studies have also now been conducted [29][30][31][32][33]. Kolossa et al showed the correspondence between surprise as the amount of information and P300 [32].…”