Preference pulses and the win–stay, fix‐and‐sample model of choice

Abstract

Two groups of six rats each were trained to respond to two levers for a food reinforcer. One group was trained on concurrent variable-ratio 20 extinction schedules of reinforcement. The second group was trained on a concurrent variable-interval 27-s extinction schedule. In both groups, lever-schedule assignments changed randomly following reinforcement; a light cued the lever providing the next reinforcer. In the next condition, the light cue was removed and reinforcer assignment strictly alternated between levers. The next two conditions redetermined, in order, the first two conditions. Preference pulses, defined as a tendency for relative response rate to decline to the just-reinforced alternative with time since reinforcement, only appeared during the extinction schedule. Although the pulse's functional form was well described by a reinforcer-induction equation, there was a large residual between actual data and a pulse-as-artifact simulation (McLean, Grace, Pitts, & Hughes, 2014) used to discern reinforcer-dependent contributions to pulsing. However, if that simulation was modified to include a win-stay tendency (a propensity to stay on the just-reinforced alternative), the residual was greatly reduced. Additional modifications of the parameter values of the pulse-as-artifact simulation enabled it to accommodate the present results as well as those it originally accommodated. In its revised form, this simulation was used to create a model that describes response runs to the preferred alternative as terminating probabilistically, and runs to the unpreferred alternative as punctate with occasional perseverative response runs. After reinforcement, choices are modeled as returning briefly to the lever location that had been just reinforced. This win-stay propensity is hypothesized as due to reinforcer induction.

Control by reinforcers across time and space: A review of recent choice research

“…Thus, preference pulses appear to reflect discriminative control by, rather than a transient response‐strengthening effect of, the most recent reinforcer location, or more probably by the location of the response that produced the previous reinforcer. Evidence supporting this has been reported by Hachiga, Sakagami, and Silberberg (; ).…”

Molecular (moment‐to‐moment) and molar (aggregate) analyses of behavior

“…There are already several kinds of simulation theories and they vary in detail, in the phenomena to which they apply, and in generality. A sample of these include those of Catania (2005), Donahoe (2010, 2013), Donahoe & Palmer (1989), Hachiga et al (2015), Li (2019), Li et al (2018a, b), Macdonall (2009), McDowell (2013), McLean et al (2014), Shimp (1969a, 1978, 1979, 1981a, 1984b, 1992, 1994), Shimp et al (1990), Shimp & Friedrich (1993), Tanno (2016), Tanno et al (2015), and Wearden & Clark (1988). Each can generate moment‐to‐moment behavior which can be compared to that of an actual organism (see Li et al, 2018a, b, for related discussion).…”

Gain-loss asymmetry in human experiential tasks

“…Our analysis showed that the unique effect of primary outcomes, when separated from the general visit structure, was to immediately reduce preference for the just-productive side (but not reverse it to the not just-productive side), which subsequently gradually increased. Our findings are preliminary given the limited literature that has applied McLean et al's (2014) correction to preference pulses (Hachiga, Sakagami, & Silberberg, 2015;Gomes-Ng et al, 2017;Gomes-Ng et al, 2018) and no application to human data to our knowledge.…”