Preference pulses induced by reinforcement

Reinforcers affect behavior. A fundamental assumption has been that reinforcers strengthen the behavior they follow, and that this strengthening may be context-specific (stimulus control). Less frequently discussed, but just as evident, is the observation that reinforcers have discriminative properties that also guide behavior. We review findings from recent research that approaches choice using nontraditional procedures, with a particular focus on how choice is affected by reinforcers, by time since reinforcers, and by recent sequences of reinforcers. We also discuss how conclusions about these results are impacted by the choice of measurement level and display. Clearly, reinforcers as traditionally considered are conditionally phylogenetically important to animals. However, their effects on behavior may be solely discriminative, and contingent reinforcers may not strengthen behavior. Rather, phylogenetically important stimuli constitute a part of a correlated compound stimulus context consisting of stimuli arising from the organism, from behavior, and from physiologically detected environmental stimuli. Thus, the three-term contingency may be seen, along with organismic state, as a correlation of stimuli. We suggest that organisms may be seen as natural stimulus-correlation detectors so that behavioral change affects the overall correlation and directs the organism toward currently appetitive goals and away from potential aversive goals. As a general conclusion, both historical and recent choice research supports the idea that stimulus control, not reinforcer control, may be fundamental.

Section: Choice Within Interreinforcer Intervalssupporting

confidence: 78%

Control by reinforcers across time and space: A review of recent choice research

Cowie

Davison

2016

“…Hachiga et al () repeated McLean et al's () simulations on their concurrent variable‐ratio (VR) extinction (EXT) schedules in order to compare simulated functional forms with those from rats in their study. They found simulated preference pulses occurred on the VR lever, a result compatible with the idea that responding in their rat data has an artifactual component.…”

mentioning

confidence: 99%

“…Hachiga et al's () rats produced a robust EXT‐lever preference pulse even though the simulation predicted its converse because rats have a time‐delimited propensity to respond to the lever that just delivered reinforcement. That tendency is present even if that lever was now signaled to be in EXT.…”

mentioning

confidence: 99%

“…As a step in this direction, this paper will first present a rudimentary model of choice which adds win–stay as a response propensity to the algorithm of McLean et al (). The model was tested against a replication and extension of the methods of Hachiga et al (). Subsequent measures will include not only the log‐P/N‐response‐ratio calculations used by Hachiga et al and other researchers of the pulsing phenomenon, but also log‐survivor plots (Shull, Gaynor, & Grimes, ), and conditional‐probability‐of‐changeover plots (Silberberg et al, ).…”

mentioning

confidence: 99%

“…Although the primary goal of the present study is to create an empirical test bed for evaluating a new model of choice, it also has two subsidiary goals: (a) to redress a few procedural problems in Hachiga et al (); and (b) to reduce differences between how the earlier paper induced pulsing versus how it has historically been done in the literature. In regard to procedural problems, this study uses a blackout following reinforcement to reduce the tendency seen in Hachiga et al for lever pressing to continue after food delivery.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Preference pulses and the win–stay, fix‐and‐sample model of choice

Hachiga

Sakagami

Silberberg

2015

Self Cite

Two groups of six rats each were trained to respond to two levers for a food reinforcer. One group was trained on concurrent variable-ratio 20 extinction schedules of reinforcement. The second group was trained on a concurrent variable-interval 27-s extinction schedule. In both groups, lever-schedule assignments changed randomly following reinforcement; a light cued the lever providing the next reinforcer. In the next condition, the light cue was removed and reinforcer assignment strictly alternated between levers. The next two conditions redetermined, in order, the first two conditions. Preference pulses, defined as a tendency for relative response rate to decline to the just-reinforced alternative with time since reinforcement, only appeared during the extinction schedule. Although the pulse's functional form was well described by a reinforcer-induction equation, there was a large residual between actual data and a pulse-as-artifact simulation (McLean, Grace, Pitts, & Hughes, 2014) used to discern reinforcer-dependent contributions to pulsing. However, if that simulation was modified to include a win-stay tendency (a propensity to stay on the just-reinforced alternative), the residual was greatly reduced. Additional modifications of the parameter values of the pulse-as-artifact simulation enabled it to accommodate the present results as well as those it originally accommodated. In its revised form, this simulation was used to create a model that describes response runs to the preferred alternative as terminating probabilistically, and runs to the unpreferred alternative as punctate with occasional perseverative response runs. After reinforcement, choices are modeled as returning briefly to the lever location that had been just reinforced. This win-stay propensity is hypothesized as due to reinforcer induction.

The logistics of choice

Killeen

2015

The generalized matching law (GML) is reconstructed as a logistic regression equation that privileges no particular value of the sensitivity parameter, a. That value will often approach 1 due to the feedback that drives switching that is intrinsic to most concurrent schedules. A model of that feedback reproduced some features of concurrent data. The GML is a law only in the strained sense that any equation that maps data is a law. The machine under the hood of matching is in all likelihood the very law that was displaced by the Matching Law. It is now time to return the Law of Effect to centrality in our science.