Neural encoding of task-dependent errors during adaptive learning

Abstract

14Learning effectively from errors requires using them in a context-dependent manner, for example 15 adjusting to errors that result from unpredicted environmental changes but ignoring errors that 16 result from environmental stochasticity. Where and how the brain represents errors in a context-17 dependent manner and uses them to guide behavior are not well understood. We imaged the 18 brains of human participants performing a predictive-inference task with two conditions that had 19 different sources of errors. Their performance was sensitive to this difference, including more 20 choice switches after fundamental changes versus stochastic fluctuations in reward contingencies. 21Using multi-voxel pattern classification, we identified context-dependent representations of error 22 magnitude and past errors in posterior parietal cortex. These representations were distinct from 23 representations of the resulting context-dependent behavioral adjustments in dorsomedial frontal, 24 anterior cingulate, and orbitofrontal cortex. The results provide new insights into human brain 25 that represent and use errors in a context-dependent manner to support adaptive behavior. 26 27 flexible learning in uncertain and dynamic environments: responding to the same exact errors 58 differently in different contexts. 59To identify such context-dependent neural responses to errors, we adapted a paradigm 60 from our previous single-unit recording study (Li et al., 2019). In this paradigm, we generated 61 two different dynamic environments by varying the amount of noise and the frequency that 62 change-points occur (i.e., hazard rate; Behrens et al.unstable environment, noise was 64 absent and the hazard rate was high, and thus errors unambiguously signaled a change in state. In 65 the high-noise environment, noise was high and the hazard rate was low, and thus small errors 66 were ambiguous and could indicate either a change in state or noise. Thus, effective learning 67 requires treating errors in the two conditions differently, including adjusting immediately to 68 errors in the unstable environment but using the size of errors and recent error history as cues to 69 aid interpretation of ambiguous errors in the high-noise condition. 70In our previous study, we found many single neurons in the anterior cingulate cortex 71 (ACC) or posterior cingulate cortex (PCC) that responded to errors or the current context, but we 72 found little evidence that single neurons in these regions combined this information in a context-73 dependent manner to discriminate the source of errors or drive behavior. In the current study, we 74 used whole-brain fMRI and multi-voxel pattern classification to identify context-dependent 75 neural responses to errors and activity predictive of context-dependent behavioral updating in the 76 human brain. The results show context-dependent encoding of error magnitude and past errors in 77 PPC and encoding of behavioral shifts in a large array of frontal regions including ACC, DMFC, 78 DLPFC and orbitofrontal cortex ...

Consistency within change: Evaluating the psychometric properties of a widely-used predictive-inference task

“…A particular challenge is the threat of poor psychometric properties of behavioural task measures, which can have considerable ramifications for both within-and between-subjects inference. In the present study, we have thus examined the psychometric properties, in particular the internal consistency and test-retest reliability, of a widely used predictive-inference task [2][3][4][6][7][8][11][12][13][14][15][16][17]33 using a large-scale, re-test online sample.…”

“…To perform well in the task, participants must adapt flexibly to these sudden changes by altering their behaviour based on the new information while ignoring the information they received before the change. Tracking participants' actions on each trial therefore allowed researchers to characterise learning 1,3,4,12,15,20 , arousal 2,6 , and neural mechanisms 7,8,11,19,21 in relation to these environmental changes. This cognitive flexibility is particularly relevant to psychiatric research, as cognitive inflexibility has been associated with several psychiatric disorders 13,18,[22][23][24][25][26][27][28][29][30][31][32] .…”

“…Despite the great popularity of this predictive-inference paradigm [2][3][4][6][7][8][11][12][13][14][15][16][17]33 , to our knowledge, the psychometric properties of the task measures have not yet been systematically investigated. In the light of an ongoing replicability crisis in the field 34 , to which inadequate psychometric qualities are a major contributor 35,36 , this is concerning, yet not surprising as it is well aligned with a general neglect of psychometric investigations of cognitive tasks 37,38 .…”

Pre-trial predictors of conflict response efficacy in human dorsolateral prefrontal cortex

“…This body of work has given rise to the “initial condition hypothesis”, which posits that neural firing patterns behave like dynamical systems, and the trajectory of neural dynamics therefore depends on the initial state of the system 3,8,35 . One implication of this theory is that preparatory neural states can therefore be optimized to efficiently produce a desired behavioral outcome 36 . If similar principles apply to neural dynamics during cognitive tasks, this suggests that preparatory activity may also be optimized to support the efficient application of cognitive control, consistent with the notion of pro-active control 37 .…”

Where Actions Meet Outcomes: Medial Prefrontal Cortex, Central Thalamus, and the Basal Ganglia

“…Lesions or inactivation of mPFC affect the ability of organisms to adapt when contingencies between stimuli, actions, and outcomes are changed during extinction learning ( Quirk and Mueller, 2008 ; Peters et al, 2009 ; Caballero et al, 2019 ; Porter and Sepulveda-Orengo, 2020 ; Green and Bouton, 2021 ; Russo et al, 2021 ). Electrophysiological recordings have identified transient post-decision signals in cingulate cortices of humans, monkeys, and rodents that indicate when rewards are received, errors are made, or expected reward not delivered ( Gemba et al, 1986 ; Ito et al, 2003 ; Botvinick et al, 2004 ; Matsumoto et al, 2007 ; Totah et al, 2009 ; Horst and Laubach, 2012 ; Li et al, 2019 ; Kao et al, 2020 ). Figure 1 shows error-related responses recorded after unreinforced incorrect choice responses in the DNMTP task.…”