Switching competitors reduces win-stay but not lose-shift behaviour: The role of outcome-action association strength on reinforcement learning

Abstract

<p>Predictability is a hallmark of poor-quality decision-making during competition. One source of predictability is the strong association between current outcome and future action, as dictated by the reinforcement learning principles of win-stay and lose-shift. We tested the idea that predictability could be reduced during competition by weakening the associations between outcome and action. To do this, participants completed a competitive zero-sum game in which the opponent from the current trial was either replayed (opponent repeat) thereby strengthening the association, or, replaced (opponent change) by a different competitor thereby weakening the association. We observed that win-stay behaviour was reduced during opponent change trials but lose-shift behaviour remained reliably predictable. Consistent with the group data, the number of individuals who exhibited predictable behaviour following wins decreased for opponent change relative to opponent repeat trials. Our data show that future actions are more under internal control following positive relative to negative outcomes, and that externally breaking the bonds between outcome and action via opponent association also allow us to become less prone to exploitation.</p>

Untitled

“…For example, in [18] (Experiment 1, no credit condition), the degree of shift behaviour is smaller following draws relative to losses (76.63% vs. 70.62%; t[39] = -2.267, p = .029; n = 40). Similar reductions in shift behaviour following draws relative to losses were also extracted from [10) (Experiment 1: 72.53% vs. 78.51%; t [35] = -2.530, p = .016; n = 36), [22] (baseline condition; 71.97% vs. 77.68%; t [35] = -2.120, p = .041; n = 36), and, [21] (70.63% vs. 75.89%; t[30] = -2.307, p = .028; n = 31). Given the attenuation of shift bias following draws relative to losses, then there is clearly some sense in which these are not identical examples of negative outcomes.…”

“…All other ANOVA main effects and interactions were not significant: cumulative score main effect [F (1,35) Shift proportions were analysed across value (win-heavy, lose-heavy),cumulative score (present, absent) and outcome (win, lose, draw) in a three-way repeated-measures ANOVA (see Fig 1). A main effect of outcome was noted [F(2,70) = 14.632, MSE = 0.077, p < .001, p 2 = .295],…”

“…Changing draw value from 0 to +1 or -1, and evaluating performance against an unexploitable opponent in Experiment 1, we make the following predictions. This is guided by data from 12 previously published experiments that use a similar win (+1), draw (0), lose (-1) outcome value assignment against an opponent who cannot be beaten [10,18,[20][21][22]35]. First, reaction times following draws should be faster than reaction times following wins (post-draw speeding).…”

Breaking the bonds of reinforcement: Effects of trial outcome, rule consistency and rule complexity against exploitable and unexploitable opponents

“…40 individuals (31 female) from the University of Sussex community participated in the study; mean age was 21.13 years (SD = 4.37) and 39 were right-handed. Sample sizes were based on previous studies from the lab showing reliable lose-shift biases within zero-sum game contexts (e.g., [ 18 ], N = 31; [ 19 ], Ns = 36; [ 21 ], Ns = 40). Participants received course credit or £10 (their choice, unless course enrollment required them to take the credit) for their participation.…”

Assessing behavioural profiles following neutral, positive and negative feedback

“…Changing draw value from 0 to +1 or -1, and evaluating performance against an unexploitable opponent in Experiment 1, we make the following predictions. This is guided by data from 12 previously published experiments that use a similar win (+1), draw (0), lose (-1) outcome value assignment against an opponent who cannot be beaten [ 10 , 18 , 20 – 22 , 35 ]. First, reaction times following draws should be faster than reaction times following wins ( post-draw speeding ).…”