Value Shapes Abstraction During Learning

Abstract

Abstractions are critical for flexible behaviours and efficient learning. However, how the brain forgoes the sensory dimension to forge abstract entities remains elusive. Here, in two fMRI experiments, we demonstrate a mechanism of abstraction built upon valuation of task-relevant sensory features. Human volunteers learned hidden association rules between visual features. Computational modelling of participants' choice data with mixture-of-experts reinforcement learning algorithms revealed that, with learning, emerging high-value abstract representations increasingly guided behaviour. Moreover, the brain area encoding value signals - the ventromedial prefrontal cortex - also prioritized and selected latent task elements, both locally and through its connection to visual cortex. In a second experiment, we used multivoxel neural reinforcement to show how reward-tagging the neural sensory representation of a task's feature evoked abstraction-based decisions. Our findings redefine the logic of valuation as a goal-dependent, key factor in constructing the abstract representations that govern intelligent behaviour.

Contributions of attention to learning in multidimensional reward environments

“…In low-dimensional environments where there are only a few features and stimuli to learn about, feature-based attention may not be required. In contrast, in high-dimensional environments in which there are Attention to reward-predictive features can be instantiated by mixtures of expert models using biologically plausible learning mechanisms (Badre and Frank, 2012;Frank and Badre, 2012;Collins and Frank, 2013;Lee et al, 2014;Cortese et al, 2021). In these models, multiple "expert" modules try to predict the reward outcome, and (approximate) Bayesian inference is used to arbitrate among these modules.…”

“…Attention to reward-predictive features can be instantiated by mixtures of expert models using biologically plausible learning mechanisms (Badre and Frank, 2012; Frank and Badre, 2012; Collins and Frank, 2013; Lee et al, 2014; Cortese et al, 2021). In these models, multiple “expert” modules try to predict the reward outcome, and (approximate) Bayesian inference is used to arbitrate among these modules.…”