Common Sense in Choice: The Effect of Sensory Modality on Neural Value Representations

Abstract

Although it is well established that the ventromedial prefrontal cortex (vmPFC) represents value using a common currency across categories of rewards, it is unknown whether the vmPFC represents value irrespective of the sensory modality in which alternatives are presented. In the current study, male and female human subjects completed a decision-making task while their neural activity was recorded using functional magnetic resonance imaging. On each trial, subjects chose between a safe alternative and a lottery, which was presented visually or aurally. A univariate conjunction analysis revealed that the anterior portion of the vmPFC tracks subjective value (SV) irrespective of the sensory modality. Using a novel cross-modality multivariate classifier, we were able to decode auditory value based on visual trials and vice versa. In addition, we found that the visual and auditory sensory cortices, which were identified using functional localizers, are also sensitive to the value of stimuli, albeit in a modality-specific manner. Whereas both primary and higher-order auditory cortices represented auditory SV (aSV), only a higher-order visual area represented visual SV (vSV). These findings expand our understanding of the common currency network of the brain and shed a new light on the interplay between sensory and value information processing.

Visual and auditory brain areas share a representational structure that supports emotion perception

“…Other possible parameters such as valence and arousal (Russell, 1980), value (Levy & Glimcher, 2012; Shuster & Levy, 2018), Fourier features (Sievers et al, 2019), HMAX (Riesenhuber & Poggio, 1999), and motion energy (Nishimoto et al, 2011) certainly covary with the stimulus feature and emotion judgment predictors. Because these and similar measures would be fully dependent on the model parameters, including them as controls would introduce collinearity and create post-treatment bias.…”

“…For example, Russell's (1980) circumplex model organizes emotions in terms of their valence (positive-negative) and degree of arousal. These and other possible parameters, such as warmth and dominance (Kiesler, 1983) or value (Levy & Glimcher, 2012;Shuster & Levy, 2018) certainly covary with the stimulus features and emotion judgments used in the model described above.…”

Attraction to similar options: The Gestalt law of proximity is related to the attraction effect

“…In perception this is known as the Weber-Fechner function [ 18 ], which states that the increase in the perceived magnitude of a certain stimulus declines as the stimulus intensity increases, and in value-based choice it is known as the Bernoulli function, or the utility function [ 19 ], which describes the notion that the marginal utility of a certain object declines as the total amount of this object increases. Second, value modulation was observed in both visual cortex [ 20 ] and auditory cortex [ 21 ] in a modality-specific way [ 22 ]. Moreover, recent studies have shown that visual selective attention is also driven by the learned value of a stimulus [ 23 , 24 ].…”

Visual and auditory brain areas share a representational structure that supports emotion perception

“…On this account, sensory regions represent conjunctions of task-relevant environmental features, such as those associated with emotion expressions, supporting direct perception of social information. 20,37 Other possible parameters, such as valence and arousal, 38 value, 39,40 Fourier features, 15 HMAX, 41 and motion energy, 42 certainly covary with the stimulus feature and emotion judgment predictors. Because these and similar measures would be fully dependent on the model parameters, including them as controls would introduce collinearity and create post-treatment bias.…”