On the futility of estimating utility functions: Why the parameters we measure are wrong, and why they do not generalize

Abstract

We have known for a long time that people’s risky choices depart systematically from expected utility theory,and also from related models like prospect theory. But it is still common to use expected utility theory orprospect theory to estimate parameters like risk aversion from sets of risky choices. We have also known fora long time that when parameters are estimated, a systematic departure between the model and the datacauses biased parameter estimates. Here we show how the bias in parameter estimation interacts with the setof choices presented to participants. We find that estimates of risk aversion vary greatly between choice setseven though no real differences in risk aversion exist. We find parameters do not generalise at all betweenchoice sets, even when the sets are random draws from a master choice set.

Reexamining How Utility and Weighting Functions Get Their Shapes: A Quasi-Adversarial Collaboration Providing a New Interpretation

“…10 Our results also hold, qualitatively, using Pearson chi-square tests for pairwise comparisons. 11 Using real choice data, Stewart et al (2018) show that the residuals resulting from playing it safe when the stakes are high are alone sufficient to create the SRH effect. They explore the broader implications of this for the nongeneralisation of utility functions estimated in different choice sets.…”

Loss aversion does disappear and reverse, although estimates of lambda (λ) are not reliable: Reply to André and De Langhe

“…Why does this happen? Stewart et al (2020) show that the differences between the statistical properties of the choice set, which occur even when choices are drawn from the same choice population, are such that behavioural departures from prospect theory cause considerable bias in the estimation of prospect theory parameters. Crucially, this bias varies considerably with the summary statistics of the gambles on offer and is large when compared to individual differences in parameters.…”

“…Here we show that André and de Langhe's measurement invariance critique of λ estimates is a special case of a much more serious model recovery problem. Stewart, Canic, and Mullett (2020) (see also Walasek & Stewart, 2020) show how fitting an incorrect model (so one from which the data systematically depart) leads to an omitted variable bias, and further, that this bias is different in choice sets spanning different parts of choice space. Stewart et al (2020) show it is futile to estimate risk aversion from choices, because the risk aversion parameter estimate does not generalise even between random splits of a large and balanced choice set.…”

“…Stewart, Canic, and Mullett (2020) (see also Walasek & Stewart, 2020) show how fitting an incorrect model (so one from which the data systematically depart) leads to an omitted variable bias, and further, that this bias is different in choice sets spanning different parts of choice space. Stewart et al (2020) show it is futile to estimate risk aversion from choices, because the risk aversion parameter estimate does not generalise even between random splits of a large and balanced choice set. Of course, all models are "wrong" to some degree as no model captures all variance in the world or perfectly predicts 100% of choices.…”

“…We take the same approach as Stewart et al (2020) fitting a version of the cumulative prospect theory (CPT) to different random subsets of the large risky choice dataset reported in Glöckner and Pachur (2012). From these fits, we ranked individual participants according to the magnitude their estimated λ based on a first, randomly selected, half of choices.…”