Adaptation of Reward Sensitivity in Orbitofrontal Neurons

Abstract

Animals depend on a large variety of rewards but their brains have a limited dynamic coding range. When rewards are uncertain, neuronal coding needs to cover a wide range of possible rewards. However, when reward is likely to occur within a specific range, focusing the sensitivity on the predicted range would optimize the discrimination of small reward differences. One way to overcome the trade-off between wide coverage and optimal discrimination is to adapt reward sensitivity dynamically to the available rewards. We investigated how changes in reward distribution influenced the coding of reward in the orbitofrontal cortex. Animals performed an oculomotor task in which a fixation cue predicted the SD of the probability distribution of juice volumes, while the expected mean volume was kept constant. A subsequent cue specified the exact juice volume obtained for a correct saccade response. Population responses of orbitofrontal neurons that reflected the predicted juice volume showed adaptation to the reward distribution. Statistical tests on individual responses revealed that a quarter of value-coding neurons shifted the reward sensitivity slope significantly between two reward distributions, whereas the remaining neurons showed insignificant change or lack of adaptation. Adaptations became more prominent when reward distributions changed less frequently, indicating time constraints for assessing reward distributions and adjusting neuronal sensitivity. The observed neuronal adaptation would optimize discrimination and contribute to the efficient coding of a large variety of potential rewards by neurons with limited dynamic range.

Normalization is a general neural mechanism for context-dependent decision making

“…Although we included a semisaturation term in the normalization algorithm to match neurophysiological observations, this parameter is not required for context dependence (SI Results); the crucial term is the value summation in the normalization denominator. In addition to normalization to the immediate choice set, the brain also displays normalization to the recent history of rewards (46)(47)(48). In theory, temporal normalization can also generate context dependence (29), but the consequences of such adaptation in value coding and potential interaction with choice circuit normalization remain currently unknown.…”

Monkeys choose as if maximizing utility compatible with basic principles of revealed preference theory

“…Thus, when presented with a set of several feasible options to which the animal revealed noisy preference or indifference, the choices from a smaller subset or larger set containing some previously chosen and unchosen elements should be similarly frequent. For several months before undergoing these tests, our monkeys had experienced a stable reward distribution, which is known to slow behavioral adaptations (36) and to render economic choices resistant to short-term adaptation (9). Such stable conditions would favor investigating the influence of option set size on preferences with little intervening adaptation to instantaneous change of option distributions.…”

Neuronal Reward and Decision Signals: From Theories to Data

“…Reward responses in orbitofrontal cortex, ventromedial prefrontal cortex, parietal cortex, striatum, globus pallidus, amygdala, lateral habenula and dopamine neurons closely reflect subjective behavioral choices (FIGURE 23) (301,405,433). Subjective value coding is also evident in specific situations, including satiation or deprivation without physical reward changes (58,105,591), differently delayed conditioned reinforcers (66,67), temporal discounting of reward value (see below FIGURE 28D) (79,158,285,477,479), and adaptation to identical reward magnitudes (see below FIGURE 33, A AND B) (42,284,332,403,601). Details will be presented in the following sections.…”

“…More formally, the adaptations may occur to changes in EV (FIGURE 33A) (42,106,232,601), variance (FIGURE 33B) (284,420,598), or both combined (80,403). In subtracting reward value from the mean (prediction error) and dividing it by standard deviation, dopamine neurons code a z-score of reward value (598).…”

“…Second, adaptations require time and may only concern subpopulations of reward neurons. Indeed, only a fraction of orbitofrontal reward neurons show adaptation, whereas the remainder codes reward magnitude irrespective of other rewards (403,406,284). Although longer test periods might have increased the number of adapting neurons, some orbitofrontal reward neurons seem to code nonadapted, absolute value necessary for maintaining transitivity.…”