Pavlovian conditioning and cumulative reinforcement rate.

Abstract

In five experiments using delay conditioning of magazine approach with rats, reinforcement rate was varied either by manipulating the mean interval between onset of the conditioned stimulus (CS) and unconditioned stimulus (US) or by manipulating the proportion of CS presentations that ended with the US (trial-based reinforcement rate). Both manipulations influenced the acquisition of responding. In each experiment, a specific comparison was made between two CSs that differed in their mean CS-US interval and in their trial-based reinforcement rate, such that the cumulative reinforcement rate-the cumulative duration of the CS between reinforcements-was the same for the two CSs. For example, a CS reinforced on 100% of trials with a mean CS-US interval of 60 s was compared with a CS reinforced on 33% of trials and a mean duration of 20 s. Across the five experiments, conditioning was virtually identical for the two CSs with matched cumulative reinforcement rate. This was true as long as the timing of the US was unpredictable, and thus response rates were uniform across the length of the CS. We conclude that the effects of CS-US interval and of trial-based reinforcement rate are reducible entirely to their common effect on cumulative reinforcement rate. We discuss the implications of this for rate-based, trial-based and real-time associative models of conditioning.

Dissociating time and number in reinforcement rate learning

“…It has been proposed that animals symbolically encode temporal durations in order to extract statistical information about events, such as the reinforcement rate during a cue's presentation 5,6 . Thus, long-duration cues elicit weaker responding than shortduration cues ('cue duration effect'), because they signal a lower rate of reinforcement 7,8 . An alternative, simple account is that time-dependent changes in stimulus processing, such as short-term habituation, lead to the effect of cue duration on learning 9,10 , with attention decreasing throughout the duration of a cue, limiting the learning that can occur.…”

Behavior Stability and Individual Differences in Pavlovian Extended Conditioning

“…Since the DOM is an extension of RW rather than a competitor, we conclude that it fits successfully 100% of the data, while the RW model accounts only for 80%. These results are strengthened when analyzing the data of Experiment 2 of Harris et al (2015), where the RW model can fit successfully only 45% of the data and the DOM 95%. Although the existence of a learning asymptote has been questioned in the past (Gallistel et al, 2004), we find its presence to be a robust feature of all the models.…”

“…We first review the RW associative model in section 2 and clarify whether it should be applied to individual subjects or to their average. In order to clarify the type of phenomena we would like to explore, in section 3.1, we present a 3,960-trial-long experiment, with a first analysis centered on the average learning curve, while in section 3.2 we reanalyze the original data of one of the preparations of Harris et al (2015). Two alternative models of individual conditioning are discussed in sections 4 (a general framework where we reformulate the RW model and introduce a new model where subjects initially show resistance to learning) and Colored Stochastic Model of Individual Behavior, where we formulate a descriptive model of short-scale random fluctuations of individual responses and contrast it with the data.…”

“…We ran an unusually long experiment with 32 rats over 3,960 trials, where we excluded habituation and other well-known phenomena as sources of variability in the subjects' performance. Using the data of this and another Pavlovian experiment by Harris et al (2015), as an illustration of the principle we tested the theory against the basic associative single-cue Rescorla-Wagner (RW) model. We found evidence that the RW model is the best non-linear regression to data only for a minority of the subjects, while its dynamical extension can explain the almost totality of data with strong to very strong evidence.…”

Rescaling of temporal expectations during extinction.

“…In contrast, studies examining changes in CS duration after partial reinforcement (PRF) focused exclusively on within-session extinction. We wondered whether PRF might encourage more temporally-based expectancies (Harris, Patterson, & Gharaei, 2015), that might render subjects more sensitive to effects of cumulative CS exposure on long-term extinction.…”