Extinction of instrumental (operant) learning: interference, varieties of context, and mechanisms of contextual control

Abstract

This article reviews recent research on the extinction of instrumental (or operant) conditioning from the perspective that it is an example of a general retroactive interference process. Previous discussions of interference have focused primarily on findings from Pavlovian conditioning. The present review shows that extinction in instrumental learning has much in common with other examples of retroactive interference in instrumental learning (e.g., omission learning, punishment, second-outcome learning, discrimination reversal learning, and differential reinforcement of alternative behavior). In each, the original learning can be largely retained after conflicting information is learned, and behavior is cued or controlled by the current context. The review also suggests that a variety of stimuli can play the role of context, including room and apparatus cues, temporal cues, drug state, deprivation state, stress state, and recent reinforcers, discrete cues, or behaviors. In instrumental learning situations, the context can control behavior through its direct association with the reinforcer or punisher, through its hierarchical relation with response-outcome associations, or its direct association (inhibitory or excitatory) with the response. In simple instrumental extinction and habit learning, the latter mechanism may play an especially important role.

Relapse of operant behavior after response elimination with an extinction or an omission contingency

“…However, there has been considerably less attention to evaluating the prevalence and extent of relapse following the end of treatment. The stronger impact of omission on reinstatement and reacquisition is consistent with the view that reinforcers delivered during omission training may provide an important part of the "context" that controls behavioral inhibition (e.g., Bouton, 2019;Bouton & Schepers, 2014;Trask, Schepers, & Bouton, 2015;Uhl & Garcia, 1969). For example, studies have found less spontaneous recovery (Davidson & Walker, 1970;Topping & Ford, 1974;Zeiler, 1971), reinstatement (Harman, 1973;Topping & Larmi, 1973;Uhl, 1973;Uhl & Garcia, 1969;Uhl & Sherman, 1971), and slower reacquisition (Mulick, Leitenberg, & Rawson, 1976;Pacitti & Smith, 1977;Uhl & Garcia, 1969;Uhl & Sherman, 1971) following omission training when compared to extinction.…”

“…For example, studies have found less spontaneous recovery (Davidson & Walker, 1970;Topping & Ford, 1974;Zeiler, 1971), reinstatement (Harman, 1973;Topping & Larmi, 1973;Uhl, 1973;Uhl & Garcia, 1969;Uhl & Sherman, 1971), and slower reacquisition (Mulick, Leitenberg, & Rawson, 1976;Pacitti & Smith, 1977;Uhl & Garcia, 1969;Uhl & Sherman, 1971) following omission training when compared to extinction. The stronger impact of omission on reinstatement and reacquisition is consistent with the view that reinforcers delivered during omission training may provide an important part of the "context" that controls behavioral inhibition (e.g., Bouton, 2019;Bouton & Schepers, 2014;Trask, Schepers, & Bouton, 2015;Uhl & Garcia, 1969). The stronger impact on spontaneous recovery, however, may be consistent with the possibility that omission creates a more durable form of behavioral suppression.…”

“…Instead, the evidence suggests that extinction involves learning to inhibit the behavior (see Bouton, 2017, 2019 for reviews). One view is that each demonstrates the fundamental context-specificity of extinction (e.g., Bouton, 2017Bouton, , 2019. The various recovery effects have been considered possible mechanisms of relapse (e.g., Bouton & Swartzentruber, 1991;Podlesnik & Shahan, 2010).…”

“…The various recovery effects have been considered possible mechanisms of relapse (e.g., Bouton & Swartzentruber, 1991;Podlesnik & Shahan, 2010). One view is that each demonstrates the fundamental context-specificity of extinction (e.g., Bouton, 2017Bouton, , 2019.…”

D1 and D2 systems converge in the striatum to update goal-directed learning

“…One of the most powerful sources of behavioral change is extinction learning, which allows individuals to withhold instrumental actions when their consequences change. Rather than erasing such actions from one's repertoire, current views propose that extinction generates new inhibitory learning that, when incorporated into previously acquired behavior, acts selectively to reduce instrumental performance (1).…”