Postreward delays and systematic biases in measures of animal temporal discounting

Abstract

Intertemporal choice tasks, which pit smaller/sooner rewards against larger/later ones, are frequently used to study time preferences and, by extension, impulsivity and self-control. When used in animals, many trials are strung together in sequence and an adjusting buffer is added after the smaller/sooner option to hold the total duration of each trial constant. Choices of the smaller/sooner option are not reward maximizing and so are taken to indicate that the animal is discounting future rewards. However, if animals fail to correctly factor in the duration of the postreward buffers, putative discounting behavior may instead reflect constrained reward maximization. Here, we report three results consistent with this discounting-free hypothesis. We find that (i) monkeys are insensitive to the association between the duration of postreward delays and their choices; (ii) they are sensitive to the length of postreward delays, although they greatly underestimate them; and (iii) increasing the salience of the postreward delay biases monkeys toward the larger/later option, reducing measured discounting rates. These results are incompatible with standard discounting-based accounts but are compatible with an alternative heuristic model. Our data suggest that measured intertemporal preferences in animals may not reflect impulsivity, or even mental discounting of future options, and that standard human and animal intertemporal choice tasks measure unrelated mental processes.delay discounting | animal models | patience | foraging | rhesus macaque A nimal decision makers in natural environments regularly face choices between smaller rewards delivered sooner and larger rewards delivered later. To study how animals make this tradeoff, psychologists often measure preferences in intertemporal choice tasks, which directly pit smaller/sooner (SS) rewards against larger/later (LL) ones (1). Animals will typically sacrifice some of the long-term rate-maximizing benefits offered by the LL option to choose SS options. These present-biased preferences are sometimes thought to reflect impulsivity and poor self-control (2-4) and are often taken as a model for human impulsivity (2-6).In nearly all animal intertemporal choice studies, many trials are strung together in sequence. When faced with a choice between an SS and an LL option, rate-maximizing behavior may sometimes dictate choosing the SS option to begin the next trial more quickly and thus increase the overall rate of reward (7). To avoid this confound, animal psychologists normally add an adjusting buffer after the SS reward to equalize trial lengths. As a result, total time for either option is matched, and the ratemaximizing strategy is to always choose the LL option. However, this buffering strategy serves its purpose only if animals correctly incorporate postreward delays into their decisions and if they correctly associate specific postreward delays with the choices that produced them. If animals fail to do either of these things, then their preferences cannot be interpreted...

Time discounting and time preference in animals: A critical review

“…Thus, delays can be costly even if they pose no special self-control problem. For this reason, several scholars have suggested that animals may avoid delays in intertemporal choice because they wish to avoid the opportunity cost they carry, and not because they lack self-control (Bateson & Kacelnik, 1996;Blanchard et al, 2013;Kacelnik, 2003;Pavlic & Passino, 2010;Stephens & Anderson, 2001). …”

“…Several authors have pointed to the postreward delay (Bateson & Kacelnik, 1996;Blanchard et al, 2013;Pavlic & Passino, 2010;Pearson, Hayden, & Platt, 2010;Stephens & Anderson, 2001). In the animal intertemporal choice task, the overall trial length is generally kept constant by lengthening the postreward delay following SS choices (Fig.…”

“…As was noted above, Ainslie did consider the possibility, but reasoned that this was unlikely because of the large amount of training they received (Ainslie, 1975). Despite this, many studies have confirmed that even well-trained animals are either less attentive or completely inattentive to any delays occurring after the reward, as compared to delays occurring before it (Bateson & Kacelnik, 1996;Blanchard et al, 2013;Goldschmidt, Lattal, & Fantino, 1998;Green, Fisher, Perlow, & Sherman, 1981;Lea, 1979;Logue, Smith, & Rachlin, 1985;Mazur, 1987Mazur, , 1989Mazur & Logue, 1978;Mazur & Romano, 1992;Mazur, Snyderman, & Coe, 1985;Pearson et al, 2010;Snyderman, 1983; see also Gallistel & Gibbon 2000). These data suggest that animals do not fully consider the postreward delay in their choices, and thus it does not fulfill its purpose.…”

“…Several pieces of evidence suggest that a failure to understand the task-what Stephens calls informational constraint-is what actually accounts for a good deal of the effect (Blanchard et al, 2013;Kacelnik, 2003;Stephens, 2002;Stephens & Anderson, 2001; for a slightly different view, see Namboodiri, Mihalas, Marton, & Hussain Shuler, 2014;Namboodiri, Mihalas, & Shuler, 2014).…”

“…A theoretical long-term rate-maximizing decision-maker with no impulsivity (i.e., an optimal forager) will prefer an SS to an LL option if the opportunity costs imposed by the LL choice outweigh the its benefits. However, this discounter may, to a naive observer, appear to be impulsive (Bateson & Kacelnik, 1996;Blanchard, Pearson, & Hayden, 2013;Kacelnik, 2003;Namboodiri, Mihalas, Marton, & Hussain Shuler, 2014;Namboodiri, Mihalas, & Shuler, 2014;Pavlic & Passino, 2010;Stephens, 2002;Stephens & Anderson, 2001). …”