Polarity of uncertainty representation during exploration and exploitation in ventromedial prefrontal cortex

Abstract

Environments furnish multiple information sources for making predictions about future events. Here we use behavioural modelling and fMRI to describe how humans select predictors that might be most relevant. First, during early encounters with potential predictors, participants' selections were explorative and directed towards subjectively uncertain predictors (positive uncertainty effect). This was particularly the case when many future opportunities remained to exploit knowledge gained. Then, preferences for accurate predictors increased over time, while uncertain predictors were avoided (negative uncertainty effect). The behavioural transition from positive to negative uncertainty-driven selections was accompanied by changes in representations of belief uncertainty in ventromedial prefrontal cortex (vmPFC). The polarity of uncertainty representations (positive or negative encoding of uncertainty) changed between exploration and exploitation periods. Moreover, the two periods were separated by a third transitional period in which beliefs about predictors' accuracy predominated. VmPFC signals a multiplicity of decision variables, the strength and polarity of which vary with behavioural context.

Effects of theta transcranial alternating current stimulation (tACS) on exploration and exploitation during uncertain decision-making

“…It was found that hemodynamic activity in the dorsolateral PFC (DLPFC) and inferior frontal gyrus (IFG), among other areas, are linearly scaled with increased reward and risk probability. Furthermore, the DLPFC has been associated with top-down processes guiding exploration and risk seeking [7,[9][10][11][12]. Additionally, it has been suggested that reward-related areas such as the anterior cingulate cortex, orbitofrontal cortex and IFG drive exploitation of safe rewards [10,[13][14][15].…”

Dissociable mechanisms of reward learning co-mature during human adolescence as predicted by macaque lesion models

“…This is in line with age-related differences of the impact that outcomes have on value updates between these age groups have been reported before (van den Bos et al ., 2012; Hauser et al ., 2015; Xia et al ., 2020). Rather than reflecting more optimal learning per se (Davidow et al ., 2016; Lloyd et al ., 2020; Lockwood et al ., 2020), the benefits of negative GRS effects, just as the ones of increased contingent learning, might be felt particularly in environments where exploration is relatively discouraged and choices should be directed towards options with high values and at the expense of sampling more uncertain options that nonetheless might prove more beneficial in the long run (Daw et al ., 2006; Wilson et al ., 2014; Trudel et al ., 2020). Our results may suggest that part of the developmental changes underlying these behaviors are rooted in a specific subcomponent of reward learning that rely on the GRS, and that is linked to late maturing lateral parts of prefrontal cortex.…”

“…By contrast, the biasing effects of irrelevant alternatives linked to medial prefrontal cortex remained unchanged across age groups. Together, this suggests that some subcomponents of reward learning in adolescence co-mature leading to overall less explorative, and more exploitative, patterns of learning and choice (Trudel et al, 2020; Wilson et al, 2014).…”

Neural activity tracking identity and confidence in social information

“…After both sessions were completed, participants completed a debriefing questionnaire probing their understanding of the task. We used data previously reported by Trudel and colleagues (2020) but now with a fundamentally different aim. Previously Trudel and colleagues had focussed on uncertainty guided exploration of reward-predicting cues regardless of whether the cues were social or non-social in nature.…”

“…Clearly it is important to learn over time whether an advice giver is predicting external events accurately. This process, sampling and learning from other people’s advice, is arguably a process that may be very similar to learning from non-social cues about the occurrence of external events (Akaishi et al, 2016; Trudel et al, 2020). In other words, the cognitive operations underlying this type of information search might occur both in social and in non-social scenarios.…”