The modulation of operant variation by the probability, magnitude, and delay of reinforcement

Abstract

Recent studies have demonstrated that the expectation of reward delivery has an inverse relationship with operant behavioral variation (e.g., Stahlman, Roberts, & Blaisdell, 2010). Research thus far has largely focused on one aspect of reinforcement – the likelihood of food delivery. In two experiments with pigeons, we examined the effect of two other aspects of reinforcement: the magnitude of the reward and the temporal delay between the operant response and outcome delivery. In the first experiment, we found that a large reward magnitude resulted in reduced spatiotemporal variation in pigeons’ pecking behavior. In the second experiment, we found that a 4-s delay between response-dependent trial termination and reward delivery increased variation in behavior. These results indicate that multiple dimensions of the reinforcer modulate operant response variation.

Midsession shifts in reward probability and the control of behavioral variability

“…By giving Shift animals a second, more salient (e.g., Donovan, 1978;Lazareva, Vecera, Levin, & Wasserman, 2005;Varela, Palacios, & Goldsmith, 1993) predictor of reinforcement probability (i.e., rather than the timing of a session's midpoint), we found strong control of behavioral variability by reinforcement probability and the elimination of the lack of difference in the first halves of sessions for Group Shift. This research supplements work from recent years on the control of variability by respondent factors (e.g., Leising et al, 2014;Stahlman & Blaisdell, 2011a). Additionally, these results are precisely what one should expect given the literature regarding midsession reversals in pigeons.…”

“…The same difference (i.e., 2 There were a number of good reasons to exclude MAX trials from the analysis, besides the nature of our a priori predictions: There were only half as many MAX trials as of either the H or the L trials; MAX trials were included merely to maintain consistency with past research and to maintain responding; and there is good reason to expect that the variability on MAX trials would not differ from variability on H trials (see Stahlman, Young, & Blaisdell 2010b). For these reasons, prior work has also excluded these trials from data analysis (Stahlman & Blaisdell 2011a). 35% vs. 4.2%) in local reinforcement probability that differentially controlled behavioral variability in the second halves of sessions did not control variability in the first halves.…”

Reward expectation modulates variability in path choice in rats

“…This potentially could indicate a role for the reinforcer obtained on Trial 1 to serve as a cue for outcomes to-beobtained on subsequent trials within a session. Taken together, these data suggest the power of discriminative stimuli to control variability based on their relationship with the outcome (e.g., Denney and Neuringer 1998;Skinner 1945;Stahlman and Blaisdell 2011a).…”

“…1 Behavior becomes more variable in circumstances where an animal could have a reduced expectation of a positive outcome (see Balsam et al 1998). This relationship has been consistently found with a number of different manipulations of the reinforcer (e.g., the delay, magnitude, and probability of a reinforcer) and in a number of different behavioral measures [e.g., bar-press duration (Gharib et al 2001(Gharib et al , 2004Roberts and Gharib 2006;Leising et al 2014), key-pecking location (Stahlman et al 2010a, b;Stahlman and Blaisdell 2011a), foraging (Stahlman and Blaisdell 2011b), nose-poke location (Antonitis 1951), and keystroke sequencing (Jensen et al 2014)]. Gharib et al (2004) proposed that behavioral variability has a negative relationship with reward expectation and that this relationship allows for adaptive flexibility in behavior.…”

Play as the Foundation of Human Intelligence: The Illuminating Role of Human Brain Evolution and Development and Implications for Education and Child Development.

“…I have also shown that the environment plays an important role in modulating behavioral variability, which may serve an important function in driving new learning about the environment (Stahlman et al, 2010a(Stahlman et al, , 2010bStahlman & Blaisdell, 2011a;2011b). Interestingly, the conditions under which variability in behavior is highest mirror those in which play behaviors are most likely to occur, when there is little expectation of an immediate outcome relevant to the individual's basic needs.…”