Learning when reward is delayed: A marking hypothesis.

Abstract

Rats were trained on spatial discriminations in which reward was delayed for 1 min. Experiment 1 tested Lett's hypothesis that responses made in the home cage during the delay interval are less likely to interfere with learning than responses made in the maze. Experimental subjects were transferred to their home cages during the delay interval, and control subjects were picked up but then immediately replaced in the maze. Contrary to Lett's hypothesis, both groups learned. Further experiments suggested that handling following a choice response was the crucial variable in producing learning: No learning occurred when handling was delayed (Experiment 2) or omitted (Experiment 3). One possible explanation for the fact that handling facilitated learning is that it served to mark the preceding choice response in memory so that subjects were then more likely to recall it when subsequently reinforced. In accordance with this interpretation, learning was found to be just as strong when the choice response was followed by an intense light or noise as by handling (Experiment 4). The implication of marking for other phenomena such as avoidance, quasi-reinforcement, and the paradoxical effects of punishment is also discussed.

Competition and facilitation in compound conditioning.

“…These results have been observed in rats using Pavlovian aversive conditioning following overshadowing (Pearce, Nicholas & Dickinson, 1981), pigeon autoshaping (Rescorla, 1982), the rabbit nictitating membrane response (Kehoe, Gibbs, Garcia, & Gormezano, 1979) and in humans as well (Reed, 1999;Shanks, 1989). Yet another phenomenon that relates to the above mentioned is "marking", a similar attenuation of trace conditioning deficits observed when a salient, albeit brief stimulus, is presented immediately after the response (Lieberman, McIntosh, & Thomas, 1979).…”

Biological constraints on instrumental and classical conditioning: Retrospect and prospect

“…Investigations of cue-consequence specificity in classical conditioning have highlighted the importance of orientation to the conditioned stimuli (Gillette, Martin, & Bellingham, 1980), similarity between the conditioned and unconditioned stimuli (e.g., Rescorla & Gillan, 1980;Testa, 1975), and temporal relationships of CS-and US-induced sensations (Krane & Wagner, 1975). Studies of longdelay learning have illustrated the importance of interfering stimuli during a CS-US interval (Lett, 1975;Revusky, 1971), of marking with a salient stimulus the response to be conditioned (Lieberman et al, 1979), and of the distinction between affective and instrumental responses (0 'Amato & Safarjan, 1981).…”

“…Recent research on "misbehavior" in depositing tokens for positive reinforcement (Timberlake, Wahl, & King, 1982), constraints on the topography of positively reinforced behavior (Shettleworth, 1975), constraints on punishment (Shettleworth, 1978), constraints on avoidance learning (Crawford & Masterson, 1978), long-delay learning (0 'Amato & Buckiewicz, 1980;Lieberman, McIntosh, & Thomas, 1979), and potentiation in classical conditioning (Pearce, Nicholas, & Dickinson, 1981) suggest that each of these phenomena may be the product of general mechanisms of learning (see Domjan, 1983). …”

Maintenance of responding when reinforcement becomes delayed

“…In contrast, when reinforcement is delayed from the outset, the leverpress has to compete with various contextual stimuli for association with reinforcement (Williams, 1999), so its associability is not greatly increased. A closely related possibility follows from the finding that discrimination learning with delayed reinforcement is enhanced if both response alternatives are "marked" (Lieberman, McIntosh, & Thomas, 1979;Lieberman & Thomas, 1986). Thus, in the present case, immediate reinforcement may "mark" the single instrumental response in a way that allows the response-reinforcer association to be maintained when the reinforcer is delayed.…”