Statistical context dictates the relationship between feedback-related EEG signals and learning

Abstract

Successful decision-making requires learning expectations based on experienced outcomes. This learning should be calibrated according to the surprise associated with an outcome, but also to the statistical context dictating the most likely source of surprise. For example, when occasional changepoints are expected, surprising outcomes should be weighted heavily, demanding increased learning. In contrast, when signal corruption is expected to occur occasionally, surprising outcomes can suggest a corrupt signal that should be ignored by learning systems. Here we dissociate surprising outcomes from the degree to which they demand learning using a predictive inference task and computational modeling. We show that the updating P300, a stimulus-locked electrophysiological response previously associated with adjustments in learning behavior, does so conditionally on the source of surprise. Larger P300 signals predicted greater learning in a changing context, but predicted less learning in a context where surprise was indicative of a one-off outlier (oddball). The conditional predictive relationship between the P300 and learning behavior was persistent even after adjusting for known sources of learning rate variability. Our results suggest that the P300 provides a surprise signal that is interpreted by downstream learning processes differentially according to statistical context in order to appropriately calibrate learning across complex environments.

Brain dynamics for confidence-weighted learning

“…It is noticeable that the modulation of evoked surprise responses was confined to early poststimulus latencies (around 200 ms) rather than occurring later, as could be expected for instance from the proposal that later brain waves like the P300 correspond to the updating, which in theory is confidence-weighted, and are enhanced by attention (Donchin, 1981;Friedman et al, 2001;Kok, 2001;Polich, 2007;Bekinschtein et al, 2009;Faugeras et al, 2012;Kolossa et al, 2013Kolossa et al, , 2015Strauss et al, 2015). However, those later brain-waves, in particular the P300, are not systematically a signature of update, for instance in a recent EEG study (Nassar et al, 2019), the P300 was modulated by surprise, but equally and irrespective of the need to update the current estimate. In line with our results, another EEG study showed that the difference between expected and unexpected sounds (standard and deviant in a oddball task) was larger around 175-200 ms (and after 350 ms) when pupil size was larger (Hong et al, 2014).…”

“…Previous studies reported conflicting results about the existence of a confidence-weighting mechanism in the brain (Jepma et al, 2016;Meyniel and Dehaene, 2017;Nassar et al, 2019). Here, we used a probability learning task previously developed, in which the participants' reports of probability estimates and the associated confidence levels are compatible with optimal Bayesian inference.…”

Confidence-controlled Hebbian learning efficiently extracts category membership from stimuli encoded in view of a categorization task

“…Theoreticians have studied different models of surprise-based learning, in the absence of reward (see [Gerstner et al, 2018] for a review). More recently, making use of fMRI [Meyniel and Dehaene, 2017], EEG [Jepma et al, 2016, Nassar et al, 2019 or MEG [Meyniel, 2019] techniques, it has been shown that this surprise signal is controlled by confidence. Confidence has been shown to grade the reward signal and impact the subsequent learning in a categorization task with mice [Lak et al, 2020].…”

Eye movements reflect adaptive predictions and predictive precision

“…Participants correctly adopted a fixed learning rate when volatility in the generative mean was governed by smooth drift (Lee et al, 2020), and downregulated their learning rate for extreme events when such events were uninformative outliers rather than true change points (D'Acremont & Bossaerts, 2016;Nassar, Bruckner, & Frank, 2019;O'Reilly et al, 2013).…”