Lever Insertion as a Salient Stimulus Promoting Insensitivity to Outcome Devaluation

Abstract

Flexible and efficient decision-making in complex environments can be achieved through constant interactions between the goal-directed and habitual systems. While goal-directed behavior is considered dependent upon Response-Outcome (R-O) associations, habits instead rely on Stimulus-Response (S-R) associations. However, the stimuli that support the S-R association underlying habitual responding in typical instrumental procedures are poorly defined. To resolve this issue, we designed a discrete-trials procedure, in which rats must wait for lever insertion and complete a sequence of five lever presses to obtain a reward (20% sucrose or grain-based pellets). Lever insertion thus constituted an audio-visual stimulus signaling the opportunity for reward. Using sensory-specific satiety-induced devaluation, we found that rats trained with grain-based pellets remained sensitive to outcome devaluation over the course of training with this procedure whereas rats trained with a solution of 20% sucrose rapidly developed habit, and that insensitivity to outcome devaluation in rats trained with sucrose did not result from a bias in general satiety. Importantly, although rats trained with pellets were sensitive to satiety-induced devaluation, their performance was not affected by degradation of instrumental contingency and devaluation by conditioned taste aversion (CTA), suggesting that these rats may also have developed habitual responding. To test whether the discrete-trials procedure biases subjects towards habitual responding, we compared discrete-trials to free-running instrumental responding, and found that rats trained with sucrose in a fixed-ratio 5 (FR5) procedure with continuous presentation of the lever were goal-directed. Together, these results demonstrate that discrete presentations of a stimulus predictive of reward availability promoted the formation of S-R habit in rats trained with liquid sucrose. Further research is necessary to explain inconsistencies in sensitivity to outcome devaluation when rats are trained with grain-based pellets.

Inflexible habitual decision-making during choice between cocaine and a nondrug alternative

“…First, we used a discretetrials procedure as opposed to free-operant schedules generally used to assess behavioral control. It was shown that rats trained in a discrete-trial procedure are prone to rapidly develop habit 53 , an effect presumably mediated by higher reinforcer predictability 54 . Secondly, we investigated sensitivity to outcome devaluation and contingency degradation in repeated reinforced choice sessions, in which rats can directly experience the devalued reward.…”

Recruitment and disruption of ventral pallidal cue encoding during alcohol seeking

“…Consistent with this, VP responses to CS cues predicting salt have been shown to track the current value of salt in the absence of opportunities for new learning (Tindell et al., ). Port entry responses to CS cues (sometimes referred to as “goal‐tracking”), which are our behavioral measure of CS learning here, have been shown to be more sensitive to outcome devaluation than approach to the CS itself (i.e., “sign‐tracking”) and may be more sensitive than behavioral responses to a DS (Colwill & Rescorla, ; Morrison, Bamkole, & Nicola, ; Vandaele, Pribut, & Janak, ). Our observation that VP responses to Pavlovian cues, but not instrumental cues, are less robust after associative or non‐associative exposure to alcohol is consistent with this framework.…”

Interval Timing and Motivation are Dissociable when Interval-Initiating Responses are Discriminable from Target Responses

“…In the present study, reinforcement followed half of the lever insertions 8 s later. Associating lever insertion with imminent reinforcement is known to promote the attribution of incentive value to levers (Beckmann & Chow, 2015;Davey & Cleland, 1982;Holland, 1977), which in turn promotes interaction with the levers and facilitation of habit development (Everitt & Robbins, 2005;Vandaele, Pribut, & Janak, 2017); habits are insensitive to manipulation by disruptors such as pre-feeding (Dickinson 1985;Vandaele, Pribut, & Janak, 2017). Given that habit development is also facilitated by extended training (Dickinson 1985;Yin & Knowlton, 2006), it is likely that trial-initiating responses were insensitive to pre-feeding because (a) in the SL-RI switch-timing procedure lever insertion was a reliable cue for both trial initiation and imminent reinforcement, and (b) the trial-initiating response was extensively trained prior to prefeeding, and was trained for longer than switch-timing.…”

“…Pre-feeding lengthened SL-RI LTIs in Experiment 2 but lengthened LTSs in SL-RI in both Experiments 1 and 2. Short-FI initiating responses in Experiment 1 likely became habitual because of an interaction between prolonged training (Dickinson, 1985) and the levers acting as time-markers (Vandaele et al, 2017), both of which have been documented to promote habitual like behavior (Vandaele, Mahajan, Ottenheimer, Richard, Mysore, & Janak, 2019;Morrison, Bamkole, & Nicola, 2015). This suggests that for LTIs to be sensitive to motivation, the interval must be initiated on manipulanda that do not also serve as time-markers.…”