Choice and Reinforcement Delay

Abstract

Previous studies of choice between two delayed reinforcers have indicated that the relative immediacy of the reinforcer is a major determinant of the relative frequency of responding. Parallel studies of choice between two interresponse times have found exceptions to this generality. The present study looked at the choice by pigeons between two delays, one of which was always four times longer than the other, but whose absolute durations were varied across conditions. The results indicated that choice is not uniquely determined by the relative immediacy of reinforcement, but that absolute delays are also involved. Models for concurrent chained schedules appear to be more applicable to the present data than the matching relation; however, these too failed to predict choice for long delays.Key words: choice, delay of reinforcement, matching law, IRTs, concurrent-chain schedules, pigeons When a reinforcer is delayed, generally it becomes less effective in strengthening or maintaining behavior. The effects of delayed reinforcement have been assessed through a variety of techniques. For example, delays have been studied with concurrent-schedule procedures where the effects due to one delay value can be assessed relative to another delay value. Chung (1965) arranged a two-key concurrentschedule-of-reinforcement procedure for pigeons under which responding on one key was immediately reinforced while responding on the other key was reinforced after a delay. The overall rates of reinforcement were equated by including blackouts on the immediate-reinforcement key. Chung found that the absolute and relative response rates on the delay key

Delay of Reinforcers in a Concurrent‐chain Schedule: An Extension of the Hyperbolic‐decay Model

“…This "failure" may not be a problem with procedures using equal-duration terminal links (see Gentry & Marr, 1980;Poniewaz, 1984; and the introduction to Experiment 2). The suggestion that, in the conventional procedure, the necessarily shorter duration of the higher reinforcer rate terminal-link schedule (hence, the more immediate production of reinforcers on subsequent trials) could add to the reinforcing value of that outcome (Mazur et al, 1985;Shull & Spear, 1987) is supported by the apparently clear different asymptotic levels of choice in the constant terminal-link duration procedure (Gentry & Marr, 1980; and the present experiment) compared with the procedure in which terminal-link delays and durations are perfectly correlated (Davison & Temple, 1973;Duncan & Fantino, 1970;Williams & Fantino, 1978). The real possibility is that, with the added reinforcement of returning to the initial link equalized between the choices, modeling concurrent-chain performance will be simplified considerably.…”

“…This discussion therefore suggests that the hyperbolic model is a serious contender for describing concurrent-chain procedures that use equal terminal-link durations. Under these conditions, the model does predict an increase in preference as the terminal-link delays are increased, and the available data (Gentry & Marr, 1980) The subjects and apparatus were the same as in Experiment 1. The procedure was the same as described in the General Procedure; the sequence of experimental conditions is shown in Table 3.…”

“…All these results suggest that there is a source of control in conventional concurrent-chain procedures that is attenuated or removed when terminal links are arranged to be equal in duration. When such procedures have been used in concurrent-chain schedules (Gentry & Marr, 1980), preference ratios did not reliably exceed reinforcer immediacy ratios even when the terminal-link durations were 128 s (cf. Duncan & Fantino, 1970 Table 2. ter reinforcers, also showed that preference still exceeded the ratio of terminal-link reinforcer immediacies.…”

“…Second, it is probably not generally applicable when terminal-link entries are arranged on dependent concurrent VI VI schedules (Fantino, 1981), although it may predict accurately when preferences are not extreme (Fantino & Royalty, 1987). Third, because delays after reinforcers in the terminal links of concurrent-chain schedules affect preference (Gentry & Marr, 1980;Poniewaz, 1984;Shull & Spear, 1987), the delay-reduction hypothesis is probably applicable only to procedures in which the terminal links end after reinforcement. Mazur's delay-of-reinforcement model has been supported only in situations in which there are fixed delays to reinforcers in the terminal links, in which relative terminal-link entry frequency is controlled (albeit normally by forced trials; but see Davison & Smith, 1986), and in which the times spent in the terminal links are equalized by arranging delays after reinforcers until the next presentation of the choice phase.…”

Choice and the Initial Delay to a Reinforcer

“…He found that preferences for the single-reinforcer terminal link were reduced when he added time after the single reinforcer (post-reinforcer delay, or PrD) so that the overall duration of the single-reinforcer terminal link equaled that of the multiple-reinforcer terminal link. this procedure was not new, since it had previously been used in Gentry and Marr (1980), Poniewaz (1984), and Snyderman (1983). Mazur found that in certain circumstances, preference for the single-reinforcer terminal link would be above 50% with no PrD, but below 50% when a PrD was added.…”