Evidence for the hierarchical structure of instrumental learning

Abstract

In three experiments using rats, we examined the role of a discriminative stimulus (8) in governing the relation between a response (R) and an outcome (0) in an appetitive instrumental learning paradigm. In each experiment, we attempted to distinguish between a simple 8-0 association and a hierarchical relation in which 8 is associated with the R-O association. We used three variations on discriminative training procedures and three different assessment techniques for revealing the hierarchical structure. In Experiment 1, we employed a training procedure in which 8 signaled a change in the R-O relation but no change in the likelihood of O. Although such an arrangement should not produce an excitatory 8-0 association, it nevertheless generated an 8 that controlled responding and transferred that control to other responses. In Experiment 2, we used a discrimination procedure in which two Ss each had the same two Rs and Os occur in their presence but each 8 signaled that a different R-O combination would be in effect. This design provided the opportunity for equivalent pairwise associations among 8, R, and 0 but unique hierarchical relations. The subjects learned the hierarchical structure, as revealed by the specific depressive effect of a subsequent lithium-chloride-induced devaluation of 0 on responding only in the presence of the 8 in which that response had led to that outcome. In Experiment 3, one 8 signaled two different R-O outcomes. Then, two new stimuli were presented with the original 8; the R-O relations were retained in the presence of one of the added stimuli but were rearranged in the presence of the other. The added 8 came to control less responding when it was redundant with respect to the R-O relations than when it was informative. Although all of the results were of modest size and each has an alternative interpretation, together they provide converging evidence for the hierarchical role of 8 in controlling an R-O association.

Pavlovian Extinction of the Discriminative Stimulus Effects of Nicotine and Ethanol in Rats Varies as a Function of Context

“…Those data were interpreted in terms of the elements that constitute the operant three-term contingency (S D : RàS r+ ): Extinction of the responseàreinforcer relationship independently from the drug states sustains the original drug (S D )àreinforcer relationship, which is only disrupted if the responseàreinforcer relationship is extinguished under the influence of the drug S D 's. These data are consistent with results of similar investigations involving exteroceptive S D 's (Colwill & Rescorla, 1990;Rescorla, 1991Rescorla, , 1994. Interestingly, no evidence for spontaneous recovery (the return of responding following delays after extinction) was observed 2 and 4 weeks following extinction of a nicotine versus saline operant discrimination; however, food pellets delivered noncontingently on responding with the levers removed from the conditioning chambers reinstated stimulus control by nicotine (Troisi, 2003a).…”

Can existing associative principles explain occasion setting? Some old ideas and some new data

“…As to whether the occasion setter acts on a specific combination of a CS and US, the evidence is mixed; while transfer studies do not support this prediction, other types of task do. In combination with findings from operant tasks examining the specificity of discriminative stimuli and inhibitors to specific response-reinforcer associations (Colwill & Rescorla, 1990;Colwill, 1991;Bonardi & Hall, 1994b), these data confirm the view that occasion setters are association-specific in their action. This pattern of findings is not anticipated by configural theory, but is consistent with the hierarchical account.…”

Negative discriminative stimuli provide information about the identity of omitted response-contingent outcomes

“…This kind of relational account is able to accommodate several experimental results that are problematical for the types of dyadic associative structures mentioned above. Especially strong evidence favoring the view that S + s encode relational information comes from a study by Colwill and Rescorla (1990) in which two S+s were trained with two responses and two outcomes. Each S+ signaled the same individual elements, but the responses and outcomes were combined in differ-ent ways so that one S+ uniquely signaled one pair of response-outcome relations andthe other S+ uniquely signaled the other pair of response-outcome relations.…”

“…Finally, it is important to acknowledge an alternative framework that has recently been proposed by Bonardi (1989) andby Colwill and Rescorla (1990) for understanding instrumental discriminative stimuli. This view attempts to represent the three-term contingency in the learning structure by allowing the S+ to activate the responseoutcome relation.…”