Contextual control of instrumental actions and habits.

Abstract

After a relatively small amount of training, instrumental behavior is thought to be an action under the control of the motivational status of its goal or reinforcer. After more extended training, behavior can become habitual and insensitive to changes in reinforcer value. Recently, instrumental responding has been shown to weaken when tested outside of the training context. The present experiments compared the sensitivity of instrumental responding in rats to a context switch after training procedures that might differentially generate actions or habits. In Experiment 1, lever pressing was decremented in a new context after either short, medium, or long periods of training on either random-ratio or yoked random-interval reinforcement schedules. Experiment 2 found that more minimally-trained responding was also sensitive to a context switch. Moreover, Experiment 3 showed that when the goal-directed component of responding was removed by devaluing the reinforcer, the residual responding that remained was still sensitive to the change of context. Goal-directed responding, in contrast, transferred across contexts. Experiment 4 then found that after extensive training, a habit that was insensitive to reinforcer devaluation was still decremented by a context switch. Overall, the results suggest that a context switch primarily influences instrumental habit rather than action. In addition, even a response that has received relatively minimal training may have a habit component that is insensitive to reinforcer devaluation but sensitive to the effects of a context switch.

Occasion setting, inhibition, and the contextual control of extinction in Pavlovian and instrumental (operant) learning

“…The results of Thrailkill and Bouton (2015b) suggesting direct control of habit by the context seem consistent with results suggesting direct inhibition of the response in extinction. However, in addition to direct excitatory or inhibitory associations with the response, contexts can also hierarchically control a response-outcome association under some conditions.…”

“…Instead, the most parsimonious explanation of the results may be that operant extinction involves direct inhibition of the response by the context. This type of learning may parallel (and functionally oppose) a related tendency for the acquisition context to evoke the response directly (Thrailkill & Bouton, 2015b; Figure 2). Although there is evidence that the context in Pavlovian and operant extinction may play other roles under some conditions, its role in Pavlovian extinction is mainly to determine the current meaning of the CS, whereas its role in operant extinction may mainly be to inhibit the response.…”

“…Thrailkill and Bouton (2015b) studied the mechanism through which the context controls the nonextinguished operant response. In their experiments, rats acquired free-operant lever press responding with relatively few (i.e., 90) or relatively many (i.e., 360) response-outcome pairings.…”

“…It also affected responding at left, but only that which remained after devaluation (responding made out of habit); it did not affect the size of the reinforcer devaluation effect (the evidence of action). Adapted from Thrailkill and Bouton (2015b).…”

A time course analysis of satiety-induced instrumental outcome devaluation

“…Therefore, the failure to observe selective devaluation after a 5 hour delay does not indicate that instrumental behavior is governed by habitual S-R processes. Instead, our results provide evidence that contextual cues may be required for the value of specific outcomes to control instrumental responding; a suggestion that is consistent with several recent reports that the context can modulate instrumental behavior (Gremel & Costa, 2013;Jonkman, Kosaki, Everitt, & Dickinson, 2010;Thrailkill & Bouton, 2015;Todd, Winterbauer, & Bouton, 2012).…”