Optimal Decision Stimuli for Risky Choice Experiments: An Adaptive Approach

Abstract

Collecting data to discriminate between models of risky choice requires careful selection of decision stimuli. Models of decision making aim to predict decisions across a wide range of possible stimuli, but practical limitations force experimenters to select only a handful of them for actual testing. Some stimuli are more diagnostic between models than others, so the choice of stimuli is critical. This paper provides the theoretical background and a methodological framework for adaptive selection of optimal stimuli for discriminating among models of risky choice. The approach, called Adaptive Design Optimization (ADO), adapts the stimulus in each experimental trial based on the results of the preceding trials. We demonstrate the validity of the approach with simulation studies aiming to discriminate Expected Utility, Weighted Expected Utility, Original Prospect Theory, and Cumulative Prospect Theory models.

On the functional form of temporal discounting: An optimized adaptive test

“…Building on these works, as well as the Bayesian adaptive design framework of Chaloner and Verdinelli (1995)), Cavagnaro et al (2010) introduced Adaptive Design Optimization as a general methodological framework for adaptive designs intended for discriminating among nonlinear mathematical models in cognitive science. The same ADO framework has since been adopted for discriminating among models of risky choice (Cavagnaro et al, 2013a), and applied to problems of discriminating among memory retention functions and among probability weighting functions (Cavagnaro et al., 2011, 2013b). Several related examples of adaptive methodologies have arisen independently in the economics and business literature, including BROAD (Ray et al, 2012), DOSE (Wang et al, 2010), and DEEP (Toubia et al, 2013).…”

“…Since the discounting models we have defined are algebraic in nature, we must equip them with a choice function, which translates utilities into choice probabilities. Following Cavagnaro et al (2013a) and Cavagnaro et al (2013b), we employ a parameter-free weak utility model that makes minimal assumptions about the structure of stochastic errors. The model assumes only that there is an unknown probability, between 0 and 0.5, of a “choice error” on any given trial, and that these choice errors are independent from trial to trial.…”

“…In an ADO experiment, stimuli are tailored to each participant by updating model and parameter estimates in real time as data are collected, and using the latest estimates to select stimuli that maximize the expected information gain about the models under consideration. The approach has proved to be effective for discriminating among models of memory retention (Cavagnaro et al, 2011) risky choice (Cavagnaro et al, 2013a) and among functional forms of the probability weighting function in Cumulative Prospect Theory (Cavagnaro et al, 2013b). The present work extends the approach to temporal discounting.…”

Risk and the Endogenous Economist: Some Comparisons of the Treatment of Risk in Physics and Economics

“…8 Of course, there is a well-known theory of stopping rules and there are some well-known solutions for well-defined problems such as the "secretary problem". For a more recent example, which will appeal to this audience, see the recent paper by Carvagnaro et al (2012). They examine a search process to find the optimal decision stimuli to test (and discriminate between) various utility-based decision models.…”

Fanning-Out or Fanning-In? Continuous or Discontinuous? Estimating Indifference Curves Inside the Marschak-Machina Triangle Using Certainty Equivalents

“…This raises a general question of how to select an optimal grid of lotteries to discriminate between the decision-making models (Cavagnaro et al, 2013), which problem is out of scope of this paper.…”