Choice Consistency and Preference Stability in Test-Retests of Discrete Choice Experiment and Open-Ended Willingness to Pay Elicitation Formats

Abstract

This study tests the temporal stability of preferences, choices and willingness to pay (WTP) values using both discrete choice experiment (DCE) and open-ended (OE) WTP elicitation formats. The same sample is surveyed three times over the course of two years using each time the same choice sets. Choice consistency is positively correlated with choice certainty and choice complexity. The impact of choice complexity fades away in time, most likely as a result of learning and preference refinement. Although the OE WTP values remain stable over a time period of 2 years as in previous stated preference studies, DCE based WTP measures differ significantly, suggesting their use in benefits transfer may be limited.

Test-Retest Reliability of Discrete Choice Experiment for Valuations of QLU-C10D Health States

“…Utility theory in health care is based on the assumption that preferences that underlie the calculation of QALYs are temporally stable [34,35] and that choices made within utility elicitation tasks are consistent [36]. As Scuffham et al [37] state, the application of QALYs in CUAs makes consideration of how they are derived and how they can be compared especially important.…”

Validity and Reliability

“…It should be noted that these also apply to comparisons of choice consistency within the same survey wave (e.g. Brouwer et al 2017;Czajkowski et al 2016), which is, however, not the focus of this section. Bowker (1948).…”

Choice certainty, consistency, and monotonicity in discrete choice experiments

“…to repeat one or several choice tasks using either the same or different samples of respondents. This is implemented between separate choice survey occasions (test-retest) which implies a time-lag (Bliem, Getzner, and Rodiga-Laßnig 2012;Brouwer, Logar, and Sheremet 2016;Liebe, Meyerhoff, and Hartje 2012;Mørkbak and Olsen 2015;Rigby, Burton, and Pluske 2016;Schaafsma et al 2014) or within the same survey which results in almost no time-lag or a lag in the order of few minutes (Brouwer et al 2010;Brown et al 2008;Campbell 2007;Carlsson, Mørkbak, and Olsen 2012;Hanley, Wright, and Koop 2002;Rigby, Burton, and Pluske 2016;Rulleau and Dachary-Bernard 2012;Scarpa, Campbell, and Hutchinson 2007;Soliño et al 2012). There is, again, a stream of literature that investigates the drivers of choice consistency in DCEs by estimating binomial models, listed in Table 2.…”

“…However, we follow the definition by Carlsson, Mørkbak, and Olsen (2012), and define choice consistency as 'making the same choice in two equal choice tasks'. An identical definition of choice consistency is used in the DCE literature on the determinants of consistency in a test-retest setting (Brouwer, Logar, and Sheremet 2016;Mørkbak and Olsen 2015;Rigby, Burton, and Pluske 2016;Schaafsma et al 2014). Finally, 'choice monotonicity' is assumed to hold if a respondent chooses a non-dominated alternative in a choice task that contains a dominated alternative, which is a hypothetical alternative that is worse than at least one other alternative in a choice task with respect to all attributes.…”