Extinction of Pavlovian conditioning: The influence of trial number and reinforcement history

Abstract

Pavlovian conditioning is sensitive to the temporal relationship between the conditioned stimulus (CS) and the unconditioned stimulus (US). This has motivated models that describe learning as a process that continuously updates associative strength during the trial or specifically encodes the CS-US interval. These models predict that extinction of responding is also continuous, such that response loss is proportional to the cumulative duration of exposure to the CS without the US. We review evidence showing that this prediction is incorrect, and that extinction is trial-based rather than time-based. We also present two experiments that test the importance of trials versus time on the Partial Reinforcement Extinction Effect (PREE), in which responding extinguishes more slowly for a CS that was inconsistently reinforced with the US than for a consistently reinforced one. We show that increasing the number of extinction trials of the partially reinforced CS, relative to the consistently reinforced CS, overcomes the PREE. However, increasing the duration of extinction trials by the same amount does not overcome the PREE. We conclude that animals learn about the likelihood of the US per trial during conditioning, and learn trial-by-trial about the absence of the US during extinction. Moreover, what they learn about the likelihood of the US during conditioning affects how sensitive they are to the absence of the US during extinction.

The Importance of Trials

“…While the results reported by Bouton et al (2014) and Chan and Harris (2017) are consistent with the prediction that a schedule that reinforces one in n trials will slow extinction by a factor of n, those particular findings rely on null results which weakens any conclusions that can be drawn from them. A stronger test of the prediction would involve comparing between a consistently reinforced CS and a partially reinforced CS, under identical extinction conditions, and measuring the number of trials taken for responding to extinguish in each case.…”

“…For example, if a CRf CS has been reinforced on every trial and a PRf CS has been reinforced on one in every 4 such trials, then it should take 4 times as much extinction exposure to the PRf CS to establish that the reinforcement rate has changed. However, this prediction has not been supported under direct test (Bouton, Woods, & Todd, 2014;Chan & Harris, 2017). Another prediction by RET is that the PREE should be eliminated if, during conditioning, the total reinforcement rate of the PRf CS is matched to that of the CRf CS.…”

“…One way to test this prediction is to give the partially reinforced CS n times as many extinction trials as the consistently reinforced CS, which should eliminate the difference in responding across extinction sessions. This was the approach taken in recent experiments by Bouton et al (2014) and Chan and Harris (2017).…”

“…For example, if the length of each conditioning trial of a CRf CS is three times longer than the length of each conditioning trial of a PRf CS reinforced on one third of trials, then the two CSs will have the same total rate of reinforcement despite a difference in how consistently they were reinforced per trial. Several groups have conducted experiments to test this prediction, and all have failed to support it (Bouton et al, 2014;Chan & Harris, 2017, 2019Harris & Kwok, 2018;Haselgrove, Aydin, & Pearce, 2004). In each case, responding to the partially reinforced CS took longer to extinguish than responding to the consistently reinforced CS.…”

The Modified Law of Effect Explains the Partial Reinforcement Extinction Effect

“…The PREE occurs, according to this theory, because it takes longer to detect the change from sparse reinforcement (PRF) to no reinforcement compared to the change from dense reinforcement (CRF) to no reinforcement. Recent attempts to test this account by equating the rate of reinforcement for CRF and PRF cues still result in the PREE, thereby challenging Gallistel's rate estimation account (Bouton, Woods, & Todd, 2014;Chan & Harris, 2017).…”