The dynamics of conditioning and extinction.

Abstract

Pigeons responded to intermittently reinforced classical conditioning trials with erratic bouts of responding to the CS. Responding depended on whether the prior trial contained a peck, food, or both. A linear-persistence/learning model moved animals into and out of a response state, and a Weibull distribution for number of within-trial responses governed in-state pecking. Variations of trial and inter-trial durations caused correlated changes in rate and probability of responding, and model parameters. A novel prediction-in the protracted absence of food, response rates can plateau above zero-was validated. The model predicted smooth acquisition functions when instantiated with the probability of food, but a more accurate jagged learning curve when instantiated with trialto-trial records of reinforcement. The Skinnerian parameter was dominant only when food could be accelerated or delayed by pecking. These experiments provide a framework for trial-by-trial accounts of conditioning and extinction that increases the information available from the data, permitting them to comment more definitively on complex contemporary models of momentum and conditioning. KeywordsAutoshaping; Behavioral momentum; Classical conditioning; Dynamic analyses; Instrumental conditioning Estes's stimulus sampling theory provided the first approximation to a general quantitative theory of learning; by adding a hypothetical attentional mechanism to conditioning, it carried analysis one step beyond extant linear learning models into the realm of theory (Atkinson & Estes, 1962;Bower, 1994;Estes, 1950Estes, , 1962Healy, Kosslyn, & Shiffrin, 1992). Wagner and Rescorla (1972) added the important nuance that the asymptotic level of conditioning might be partitioned among stimuli that are associated with reinforcers, as a function of their reliability as predictors of reinforcement; that refinement has had tremendous and widespread impact (Siegel & Allan, 1996). The attempt to couch the theory in ways that account for increasing amounts of the variance in behavior has been one of the main engines driving modern learning theory. Models have been the agents of progress, the go-betweens that reshaped both our theoretical inferences about the conditioning processes, and our modes of analysis of the data. In this theoretical-empirical dialog, the Rescorla-Wagner (R-W) model has been paragon.Despite the elegant mathematical form of their arguments, the predictions of recent learning models are almost always qualitative-a particular constellation of cues is predicted to block or enhance conditioning more than others, due to their differential associability, or their history Correspond with: Peter Killeen, Department of Psychology, Arizona State University, Box 871104, Tempe, Arizona 85287-1104, USA, Killeen@asu.edu. Publisher's Disclaimer: The following manuscript is the final accepted manuscript. It has not been subjected to the final copyediting, fact-checking, and proofreading required for formal publication. It is not the definitive, publis...

Repeated extinction and reversal learning of an approach response supports an arousal-mediated learning model

“…Killeen et al (2009) suggested a dynamic model of conditioning and extinction similar to the arousal-mediated learning model advanced here, but without the arousal component. In their Experiment 1, when rate of reinforcement varied across conditions, parameter estimates changed; in their Experiment 2, when rate of reinforcement was constant (only the CS duration varied) parameter estimates remained constant.…”

“…Extinction of the conditional stimulus is a learning process, critical to the organism's adaptation to a changing environment. There has been much conceptual and theoretical development devoted toward elucidating processes mediating declines in conditioned performance during extinction (e.g., Bouton, 2004; Gallistel and Gibbon, 2000; Killeen et al, 2009; Mackintosh, 1975; Pearce and Hall, 1980; Rescorla, 2001; Rescorla and Wagner, 1972; Wagner, 1981). One development that has become almost universally accepted is that declines in responding are not a result of extinction eliminating prior excitatory learning (e.g., Rescorla, 1993).…”

“…The model is based on Killeen et al's (2009) notion that the probability of responding to a CS on any given trial is a function of past CS-US pairings – much in line with classic dynamic learning models and even earlier stimulus sampling models (Estes, 1950) – and the momentum of response and no-response states. The new model explains the persistence of learning using just one additional assumption, that learning and performance are conditional to US-elicited arousal (Killeen et al, 1978).…”

Extinction under a behavioral microscope: Isolating the sources of decline in operant response rate

“…The term is often used interchangeably to refer to the change in response rate and to the discontinuation of reinforcement. The performance component of extinction—the change in response rate—is of theoretical interest, because it may express critical learning and memory processes (Bouton 2004; Killeen et al 2009). The procedural component of extinction—the discontinuation of reinforcement—is often implemented to test the effects of treatments on behavior without the interference of reinforcement.…”

Adjunctive behaviors are operants

“…Their report was interpreted as showing that Pavlovian contingencies that fostered auto-shaped pecking were dominant over the Skinnerian contingencies that discouraged it. Nonreinforcement has a much smaller "corrective effect" than does reinforcement (Killeen, Sanabria & Dolgov, 2009): Terminal responses can be maintained by very thin payoff ratios, with thousands of responses maintained by a single reinforcer. But it does have an effect: Under extended exposure to the Williams and Williams's procedure, keypecking extinguishes (Sanabria, Sitomer & Killeen, 2006).…”