Reanalysis of Butler and Pogrebna (2018) using true and error model

Abstract

Butler and Pogrebna (2018) devised triples of three-branch gambles theorized to violate transitivity of preference according to a most probable winner model. According to this model, a person chooses the option that has the higher probability to yield a better outcome than the other alternative. They tested 11 triples with 100 participants and found cases that appeared to violate weak stochastic transitivity and the triangle inequality. But tests of weak stochastic transitivity and the triangle inequality do not provide a proper method to compare transitive and intransitive models that allow mixtures of preference patterns and random errors. Those older methods can yield false conclusions regarding transitivity, for example, if different participants have different true preferences or if different choice problems have different rates of error. This paper reanalyzes their data using a true and error (TE) model, which does not require these unrealistic assumptions, and which provides estimates of the incidence of transitive and intransitive behavior in a mixture. Reanalysis indicated that 3 of the 11 triples showed convincing evidence of violations of transitivity in the opposite direction of the predictions of the most probable winner model. Further, these and other triples showed other significant violations of the most probable winner model. Despite some violations of the true and error model, the data of Butler and Pogrebna appear to contradict not only transitive utility models but they also refute the most probable winner model as a descriptive theory of choice behavior.

Testing transitivity of preference in individuals.

“…Four of the 11 triples had estimated incidences of intransitive behavior that were statistically significant, according to the TE analysis. Birnbaum (2020) and Butler (2020) agreed that the stimuli of Butler and Pogrebna (2018) had generated systematic evidence of violation of transitivity and that this design should be pursued in further investigations of this property.…”

“…A reanalysis of the Butler and Pogrebna data using a true and error (TE) model found that there was modest, but statistically significant evidence of systematic violations of transitivity (Birnbaum, 2020): It was estimated that 11% of the preference patterns were compatible with MPW, and about 18% were intransitive preferences of the opposite type. Four of the 11 triples had estimated incidences of intransitive behavior that were statistically significant, according to the TE analysis.…”

MARTER: Markov True and Error model of drifting parameters

“…Müller-Trede (personal communication, Jan. 3, 2020) reanalyzed the data of Müller-Trede, et al (2015, Experiment 1) and found that 5 of 22 participants had estimates of probability of the predicted intransitive pattern significantly exceeding 0; for these same 5, the authors had rejected the triangle inequality. Birnbaum (2020) reanalyzed data from Butler and Pogrebna (2018) and found that 3 of the 11 triples they tested showed significant probabilities of the intransitive preference pattern predicted by the most probable winner model, with rates of this violation ranging from 34% to 51% in the three cases that were significant.…”

Intransitive preferences or choice errors? A reply to Birnbaum

“…

Birnbaum (2020) reanalyses the data from Butler and Pogrebna (2018) using his 'true and error' test of choice patterns. His results generally support the evidence we presented in that paper.

…”

“…It would be useful to understand how to apply or extend the true and error model for the comparisons between an individual's binary and ternary choices, as that is a key part of our study of intransitivity in the paper. There are two other observations we would like to make regarding Birnbaum (2020). The first concerns his remarks on the evidence for most probable winner (MPW) cycles, and the second, on the significance of predictably intransitive choice patterns.…”