Electrophysiological correlates reflect the integration of model-based and model-free decision information

Abstract

In this study, we investigated the interplay of habitual (model-free) and goal-directed (model-based) decision processes by using a two-stage Markov decision task in combination with event-related potentials (ERPs) and computational modeling. To manipulate the demands on model-based decision making, we applied two experimental conditions with different probabilities of transitioning from the first to the second stage of the task. As we expected, when the stage transitions were more predictable, participants showed greater model-based (planning) behavior. Consistent with this result, we found that stimulus-evoked parietal (P300) activity at the second stage of the task increased with the predictability of the state transitions. However, the parietal activity also reflected model-free information about the expected values of the stimuli, indicating that at this stage of the task both types of information are integrated to guide decision making. Outcome-related ERP components only reflected reward-related processes: Specifically, a medial prefrontal ERP component (the feedback-related negativity) was sensitive to negative outcomes, whereas a component that is elicited by reward (the feedback-related positivity) increased as a function of positive prediction errors. Taken together, our data indicate that stimulus-locked parietal activity reflects the integration of model-based and model-free information during decision making, whereas feedback-related medial prefrontal signals primarily reflect reward-related decision processes.

Electrophysiological correlates of state transition prediction errors

“…By contrast, Experiment 1 in our study eliminated the RPE/SPE confound in the two-step task by removing the reward element, which revealed a robust SPE. Similarly, Eppinger et al (2017) reported an effect of transition type on the LPC that replicates our ERP findings reported in the SOM, though their effect was considerably smaller (dz=0.48) relative to that of our study (dz=1.17; supplementary online material). In summary, whereas our findings do not generally disagree with the previous results, ours is the first to report that SPEs elicit a robust FMT response and elevated delta levels that correlate with behavioral adjustments.…”

Model-based learning deficits in compulsivity are linked to faulty representations of task structure

“…Previous EEG studies of the two-step task (Eppinger et al, 2017;Sambrook et al, 2018;Shahnazian et al, 2019) showed that the P300 was associated with state transitions. However, the inconsistent direction on the effects raises doubt as to how these differences should be interpreted.…”

“…Participants have to take into consideration the task transition structure and their history of rewards to make choices that maximise reward. The sequence of events as presented for EEG is the same as that of Eppinger et al (2017), except they included a manipulation of transition probabilities in their study (comparing 60/40% to 80/20%) and used a longer choice window (2000ms).…”

“…The P300 or P3b has well-established sensitivity to stimulus probability, exhibiting larger peak amplitudes for less probable stimuli (Polich & Margala, 1997). Prior research in healthy humans thus hypothesised that the P300 may be a marker of sensitivity to state transitions on the two-step task, though these studies have yielded inconsistent results, with some finding greater P300 amplitudes for rare versus common transitions (Sambrook, Hardwick, Wills, & Goslin, 2018;Shahnazian, Ribas-Fernandes, & Holroyd, 2019) and one finding the opposite (Eppinger, Walter, & Li, 2017). Here, we examined the second stage stimulus-locked P300 and found a significant main effect of transition type (β = 0.15, SE = 0.07, p = 0.03), consistent with Sambrook et al (2018) and Shahnazian et al (2019) whereby greater P300 amplitude was observed after rare versus common transitions (Figure 3).…”

Neural processing of iterated prisoner’s dilemma outcomes indicates next-round choice and speed to reciprocate cooperation

“…In value-based decision making, the P3 is part of feedback processing 25,27,49 , and shows sensitivity to reward magnitude, valence and probability/risk [50][51][52][53][54][55][56][57][58] . As in perceptual studies, the feedback P3 may reflect update and storage of reward values for behavioral adjustment 25,[59][60][61][62] , by integrating them with internal models of a probabilistic task 63 . It may similarly reflect reward evidence accumulation dynamics 64 , possibly as a decision variable for strategic choice 65 .…”