The effect of reinforcement probability on time discrimination in the midsession reversal task

Abstract

We examined how biasing time perception affects choice in a midsession reversal task. Given a simultaneous discrimination between stimuli S1 and S2, with choices of S1 reinforced during the first, but not the second half of the trials, and choices of S2 reinforced during the second, but not the first half of the trials, pigeons show anticipation errors (premature choices of S2) and perseveration errors (belated choices of S1). This suggests that choice depends on timing processes, on predicting when the contingency reverses based on session duration. We exposed 7 pigeons to a midsession reversal task and manipulated the reinforcement rate on each half of the session. Compared to equal reinforcement rates on both halves of the session, when the reinforcement rate on the first half was lower than on the second half, performance showed more anticipation and less perseveration errors, and when the reinforcement rate on the first half was higher than on the second half, performance showed a remarkable reduction of both types of errors. These results suggest that choice depends on both time into the session and the outcome of previous trials. They also challenge current models of timing to integrate local effects.

Step changes in the intertrial interval in the midsession reversal task: Predicting pigeons' performance with the learning‐to‐time model

“…5 for examples). These performance idiosyncrasies are consistent with previous reports (Rayburn-Reeves and Cook, 2016;Santos et al, 2019) and might require further research.…”

“…Whence the asymmetry? The results from this and similar studies (McMillan & Roberts, 2012; Santos et al, 2019; Smith et al, 2017, Zentall et al, 2019) indicate that pigeons' performance in the MSR task is not exclusively controlled by timing. Local cues, including the response outcome of previous trials, may also influence choice on the next trial.…”

“…The links from each state to S1 and S2 were initialized to W 0 = .5. The values of W 0 and β were similar to those used in a previous study (Santos et al, 2019), but whereas Santos et al (2019) adapted α, μ λ and σ λ to each pigeon, we maintained these parameters also constant because we wanted to compare the data with the model predictions based exclusively on the ITI changes. We used a time step of 1 s and assumed that the latency to choose the stimuli always equaled 1 s.…”

“…LeT aims to describe how animals learn to regulate their behavior in time‐dependent reinforcement schedules, including fixed‐interval schedules, peak procedure, temporal bisection, temporal generalization, and the free‐operant psychophysical procedure (for reviews see Carvalho et al, 2016; Machado et al, 2009). Because LeT is a dynamic, real‐time model, it can be exposed also to seemingly nontemporal tasks such as the MSR task (Santos et al, 2019). In those tasks, LeT can play the useful role of a standard, showing not only how an animal might learn the temporal regularities embeded in the task—in this case, when the contingency reverses during the session—but also how choices based exclusively on the temporal discrimination, with its uncertainty (Weber's law), might distribute across a session, engendering anticipatory and perseverative errors, and how the animal might react to a temporal manipulation such as an ITI change.…”

The Case for a Heuristic Approach to Account for Suboptimal Choice

“…But as we have shown (Rayburn-Reeves, Molet, & Zentall, 2011), pigeons respond to the feedback from an unpredictable reversal much more slowly when it occurs early in the session than when it occurs later in the session. Carvalho et al (2019) question my interpretation of why reducing the probability of reinforcement for correct S2 responses (from 100% to 20%) reduces errors (especially anticipatory errors) but reducing the probability of reinforcement for correct S1 responses (from 100% to 20%) actually increases errors (especially anticipatory errors; Santos, Soares, Vasconcelos, & Machado, 2019). In my target article, I suggested that reducing the probability of reinforcement for correct S2 responses from 100% to 20% reduces the response competition between S1 and S2 and encourages the pigeons to choose based on the consequences of choice of S1 alone.…”