The behavioral economics of choice and interval timing.

Abstract

We propose a simple behavioral economic model (BEM) describing how reinforcement and interval timing interact. The model assumes a Weber-law-compliant logarithmic representation of time. Associated with each represented time value are the payoffs that have been obtained for each possible response. At a given real time, the response with the highest payoff is emitted. The model accounts for a wide range of data from procedures such as simple bisection, metacognition in animals, economic effects in free-operant psychophysical procedures and paradoxical choice in doublebisection procedures. Although it assumes logarithmic time representation, it can also account for data from the time-left procedure usually cited in support of linear time representation. It encounters some difficulties in complex free-operant choice procedures, such as concurrent mixed fixed-interval schedules as well as some of the data on double bisection, that may involve additional processes. Overall, BEM provides a theoretical framework for understanding how reinforcement and interval timing work together to determine choice between temporally differentiated reinforcers. Keywordsbehavioral economic model; choice; interval timing; reinforcement; Weber's law Choice and interval timing are two important areas of operant behavior that have remained relatively isolated from one another. Following the lead of B. F. Skinner, (Skinner, 1938;Ferster & Skinner, 1957), research on choice has focused on the role of "economic" variables (frequency and magnitude of reinforcement, etc.; see Williams, 1988 for a review) while research on timing was approached early on from a psychophysical point of view (Catania, 1970;Dews, 1970;Gibbon, 1977) which emphasized cognitively oriented questions such as the representation of time and the information-processing mechanisms governing temporal regulation of behavior (see Church, 2004;Staddon & Cerutti, 2003;Jozefowiez & Staddon, 2008, for some recent reviews). Choice researchers have ignored cognitive questions, while at the same time students of timing have underplayed the role of reinforcement. Yet it is clear that a complete theory of operant performance will require an integration of these two fields of research (see also Whitaker, Lowe, & Wearden, 2008 NIH-PA Author ManuscriptNIH-PA Author Manuscript NIH-PA Author ManuscriptInterval timing, the ability of animals to adapt to temporal relations ranging from seconds to minutes between two events, has been observed in a wide range of species, from fish to humans (Lejeune & Wearden, 1991). In pigeons at least, it has an automatic, almost reflex-like nature (Wynne & Staddon, 1988), suggesting strong selection pressure at an evolutionary level, hence a potentially key role in adaptive behavior. Indeed, according to some recent accounts (Gallistel & Gibbon, 2000;Staddon & Cerutti, 2003;Staddon & Ettinger, 1989;Cerutti & Staddon, 2004b;Grace, Berg, & Kyonka, 2006;Shapiro, Miller, & Kacelnik, 2008), interval timing may play an important role in choice. Moreover, s...

Stimulus probability effects on temporal bisection performance of mice (Mus musculus)

“…As discussed by Jozefowiez et al (2014), within the Scalar Expectancy Theory framework (Gibbon et al 1984), changes in the PSEs would be dependent on the memory representations of reference durations as well as a parameter representing response bias (Gibbon 1981). Similarly, the Behavioral Economic Model (Jozefowiez et al 2009) would also incorporate a term accounting for the bias arising specifically from the manipulations of reinforcement probability and frequency of target durations into temporal decisions based on the associative strength between the timed interval and short and long responses (Jozefowiez et al 2014). Although these models differ in their definitions of the underlying decision rules and predictions regarding the magnitude of bias manifested in the temporal bisection performance (Jozefowiez et al 2014), they both attribute the stimulus probability effects on the PSEs to a bias parameter.…”

Adjunctive behaviors are operants

“…The resulting models are parsimonious of parameters, even if the hypothetical construct-the traces-are inferred. A similar mechanism, involving competition between temporally privileged concurrent actions, has been posited for instrumental responses by Jozefowiez et al (2009).…”

“…4. There are many timing models that do this (e.g., Jozefowiez, Staddon, & Cerutti, 2009). Most congenial to the present account are those that posit transitions from one behavioral state to the next (Killeen & Fetterman, 1993)-for instance, the learning-to-time model (Machado, 1997) and the stochastic counter model (Killeen, 2002;Killeen & Taylor, 2000a, b), which preserve essential features of timing, such as scalar invariance.…”

Trial frequency effects in human temporal bisection: Implications for theories of timing

“…On the other hand, some other researchers have viewed the problem the other way round and proposed that associative learning underlies interval timing. This is, for instance, the case in Machado's Learning-to-Time model (LeT) model (Machado, 1997; Machado, Malheiro, & Erlhagen, 2009) and Jozefowiez, Staddon, and Cerutti's (2009) Behavioral Economic Model (BEM), in which the learning of associations between internal, time-dependent states and responding results in interval timing (see also Ludvig, Sutton, & Kehoe, 2012). Those associations are supposed to follow the general principles of conditioning that are embedded in classical models of learning such as the Bush-Mosteller integrator (Bush & Mosteller, 1955; Stout & Miller, 2007) or the Rescorla-Wagner model (Rescorla & Wagner, 1972).…”

“…If R(S) (or R[L]) is emitted while the subject is in state x, the association between state x and R(S) (or R[L]) changes according to a linear operator rule (e.g., Bush & Mosteller, 1955), increasing in the case of reinforcement and decreasing in the case of nonreinforcement. Moreover, BEM assumes a logarithmic encoding of time: The active state at interval t, x(t), is a random variable drawn from a Gaussian distribution with mean ln t and standard deviation k (see Jozefowiez et al, 2009, in press, for arguments justifying the choice of a log representation over a linear representation; see also Dehaene, 2001; Roberts, 2008 and Yi, 2008 for theoretical and empirical arguments favoring a log over a linear representation). We call the random variable x the short-term memory (STM) representation of an interval t. In addition, let V1(x) be the strength of the association between x and R(S) and V2(x) be the strength of the association between x and R(L).…”