Choice in a Self‐control Paradigm: Quantification of Experience‐based Differences

Abstract

Previous quantitative models of choice in a self-control paradigm (choice between a larger, more-delayed reinforcer and a smaller, less-delayed reinforcer) have not described individual differences. Two experiments are reported that provide additional quantitative data on experience-based differences in choice between reinforcers of varying sizes and delays. In Experiment 1, seven pigeons in a self-control paradigm were exposed to a fading procedure that increased choices of the larger, more-delayed reinforcer through gradually decreasing the delay to the smaller of two equally delayed reinforcers. Three control subjects, exposed to each of the small-reinforcer delays to which the experimental subjects were exposed, but for fewer sessions, demonstrated that lengthy exposure to each of the conditions in the fading procedure may be necessary in order for the increase to occur. In Experiment 2, pigeons with and without fading-procedure exposure chose between reinforcers of varying sizes and delays scheduled according to a concurrent variable-interval variable-interval schedule. In both experiments, pigeons with fading-procedure exposure were more sensitive to variations in reinforcer size than reinforcer delay when compared with pigeons without this exposure. The data were described by the generalized matching law when the relative size of its exponents, representing subjects' relative sensitivity to reinforcer size and delay, were grouped according to subjects' experience.

Sensitivity of pigeons to prereinforcer and postreinforcer delay

“…Despite this extensive exposure to a choice between a larger reinforcer with a longer prereinforcer delay and a smaller reinforcer with a shorter prereinforcer delay, with forced choices of each alternative, virtually none of the larger reinforcers were chosen throughout these 8 conditions. Previous data have indicated that exposure to very gradual approximations of this choice, beginning with a choice between larger and smaller reinforcers with equal prereinforcer delays, is necessary for pigeons to acquire selfcontrol (Logue et al, 1984;Mazur & Logue, 1978).…”

“…Postreinforcer delay was varied both while holding rate of reinforcer access constant and while allowing it to vary. Previous experiments, which showed no control by postreinforcer delays when prereinforcer delays were varied, used nondifferential postreinforcer stimuli for the two alternatives (see Logue et al, 1984;Mazur & Logue, 1978). Consequently, stimuli associated with the most recent choice that had been made were present during the postreinforcer delay.…”

Assessment of Impulsivity and the Development of Self‐control in Students With Attention Deficit Hyperactivity Disorder

“…This study therefore extends behavioral investigations of self-control (e.g., Dixon et al, 1998;Dixon & Holcomb, 2000;Schweitzer & SulzerAzaroff, 1988) by considering choices for delayed reinforcers in the context of dimensions in addition to the rate or amount of reinforcement (e.g., forgoing immediate re-inforcers in favor of later access to those that are more highly valued or preferred). Basic research has suggested that fading procedures increase sensitivity to reinforcer amounts relative to reinforcer delays (Logue et al, 1984), and this might also apply to other dimensions such as reinforcer quality.…”

“…Research with both human and nonhuman animals has shown that self-control can be developed by gradually increasing the delay to the larger reinforcer (Dixon et al, 1998;Logue, Rodriguez, Peña-Correal, & Mauro, 1984;Mazur & Logue, 1978;Ragotzy, Blakely, & Poling, 1988;Schweitzer & Sulzer-Azaroff, 1988). Dixon et al, for example, established self-control in 3 adults with developmental disabilities by first making both the smaller and larger reinforcers for desired behaviors available immediately, and then progressively increasing the delay for the schedule associated with the larger reinforcer only.…”

Decision Making Across the Life Span

“…Reinforcement learning models developed to capture these data are notoriously slow to learn about delayed outcomes (Sutton and Barto, 1998). As a consequence it may take substantial time and experience (Logue et al, 1984) for mesolimbic dopamine regions to develop robust responses to cues predicting rewards at long time delays. Although older adults may suffer from declines in fluid cognitive ability that may constrain their decision making competence, they also have decades of experience over their young adult counterparts with the realization of delayed rewards which may lead to similar decisions behaviorally (Agarwal et al, 2009).…”