Sign-tracking rats show heightened sensitivity to food- and drug-associated cues, which serve as strong incentives for driving reward seeking. We hypothesized that this enhanced incentive drive is accompanied by an inflexibility when incentive value changes. To examine this we tested rats in Pavlovian outcome devaluation or second-order conditioning prior to the assessment of sign-tracking tendency. To assess behavioral flexibility we trained rats to associate a light with a food outcome. After the food was devalued by pairing with illness, we measured conditioned responding (CR) to the light during an outcome devaluation probe test. The level of CR during outcome devaluation probe test correlated with the rats' subsequent tracking tendency, with sign-tracking rats failing to suppress CR to the light after outcome devaluation. To assess Pavlovian incentive learning, we trained rats on first-order (CS+, CS−) and second-order (SOCS+, SOCS−) discriminations. After second-order conditioning, we measured CR to the second-order cues during a probe test. Second-order conditioning was observed across all rats regardless of tracking tendency. The behavioral inflexibility of sign-trackers has potential relevance for understanding individual variation in vulnerability to drug addiction.

Phasic activity of midbrain dopamine neurons is currently thought to encapsulate the prediction-error signal described in Sutton and Barto’s (1981) model-free reinforcement learning algorithm. This phasic signal is thought to contain information about the quantitative value of reward, which transfers to the reward-predictive cue after learning. This is argued to endow the reward-predictive cue with the value inherent in the reward, motivating behavior toward cues signaling the presence of reward. Yet theoretical and empirical research has implicated prediction-error signaling in learning that extends far beyond a transfer of quantitative value to a reward-predictive cue. Here, we review the research which demonstrates the complexity of how dopaminergic prediction errors facilitate learning. After briefly discussing the literature demonstrating that phasic dopaminergic signals can act in the manner described by Sutton and Barto (1981), we consider how these signals may also influence attentional processing across multiple attentional systems in distinct brain circuits. Then, we discuss how prediction errors encode and promote the development of context-specific associations between cues and rewards. Finally, we consider recent evidence that shows dopaminergic activity contains information about causal relationships between cues and rewards that reflect information garnered from rich associative models of the world that can be adapted in the absence of direct experience. In discussing this research we hope to support the expansion of how dopaminergic prediction errors are thought to contribute to the learning process beyond the traditional concept of transferring quantitative value.

The existence of value coding and salience coding neurons in the mammalian brain, including in habenula and ventral tegmental area, has sparked considerable interest in the interactions that occur between Pavlovian appetitive and aversive conditioning. Here we studied these appetitive-aversive interactions at the behavioral level by assessing the learning that occurs when a Pavlovian appetitive conditioned stimulus (conditional stimulus, CS) serves as a CS for shock in Pavlovian fear conditioning. A Pavlovian appetitive CS was retarded in the rate at which it could be transformed into a fear CS (counterconditioning), but the presence of the appetitive CS augmented fear learning to a concurrently presented neutral CS (superconditioning). Retardation of fear learning was not alleviated by manipulations designed to restore the associability of the appetitive CS before fear conditioning but was alleviated by manipulations designed to increase the aversive quality of the shock unconditioned stimulus (US). These findings are consistent with opponent interactions between the appetitive and aversive motivational systems and provide a behavioral approach for assessing the neural correlates of these appetitive-aversive interactions.