Encoding of continuous perceptual choices in human early visual cortex

Abstract

Research on the neural mechanisms of perceptual decision-making has typically focused on simple categorical choices, say between two alternative motion directions. Studies on such discrete alternatives have often suggested that choices are encoded either in a motor-based or in an abstract, categorical format in regions beyond sensory cortex. However, many sensory features are graded rather than discrete, raising the question how choices are encoded when they span the full sensory continuum. Here we assessed this using motion stimuli that could vary anywhere between 0° and 360°. We employed a combination of neuroimaging and encoding models based on Gaussian Process Regression to assess how either stimuli or choices were encoded in brain responses. We found that single-voxel tuning patterns could be used to reconstruct the trial-by-trial physical direction of motion as well as the participants' continuous choices. Importantly, these continuous choice signals were primarily observed in early visual areas. The tuning properties in this region generalized between choice encoding and stimulus encoding, even for reports that reflected pure guessing. We found only little information related to the decision outcome in regions beyond visual cortex, such as parietal cortex, possibly because our task did not involve differential motor preparation. This could suggest that decisions for continuous stimuli take can place already in sensory brain regions, potentially using similar mechanisms to the sensory recruitment in visual working memory.

Working memory signals in early visual cortex are present in weak and strong imagers

“…In this study, we used nested cross‐validation across subjects to determine the optimal kernel width for each participant (see below). Please note that all these approaches for temporal detrending and feature‐space smoothing were developed and optimized on a separate data set (from a related study; Barbieri et al, 2023) and both were preregistered and checked for artifacts or spurious effects.…”

“…BFCA is an extension of the concept of balanced accuracy (Brodersen et al, 2010) for continuous variables. It is calculated by computing the integral of the trial‐wise FCA from 0° to 180° (i.e., the orientation‐space), using trapezoidal numerical integration across the sorted true and reconstructed orientations: (Barbieri et al, 2023) $$$$ \mathrm{BFCA}=\frac{1}{180}\ \int_0^{180}\mathrm{FCA}\left(\theta, \hat{\theta}\right)\ \mathrm{d}\theta . $$$$ …”

“…BFCA is an extension of the concept of balanced accuracy (Brodersen et al, 2010) for continuous variables. It is calculated by computing the integral of the trial-wise FCA from 0 to 180 (i.e., the orientation-space), using trapezoidal numerical integration across the sorted true and reconstructed orientations: (Barbieri et al, 2023) BFCA ¼ 1 180…”

Working memory signals in early visual cortex do not depend on visual imagery

“…It is calculated by computing the integral of the trial-wise FCA from 0° to 180° (i.e. the orientation-space), using trapezoidal numerical integration across the sorted true and reconstructed orientations (Barbieri et al, 2023): …”

“…The preregistration was submitted after data acquisition, but prior to data processing and analysis. All preregistered analysis procedures were developed and/or optimized on a separate fMRI dataset from a related study (Barbieri et al, 2023). Please note that we did not change any of the preregistered workflows.…”