A Picture is Worth a Thousand Words: The Role of Survey Training Materials in Stated-Preference Studies

Abstract

Background Online survey-based methods are increasingly used to elicit preferences for healthcare. This digitization creates an opportunity for interactive survey elements, potentially improving respondents' understanding and/or engagement. Objective Our objective was to understand whether, and how, training materials in a survey influenced stated preferences. Methods An online discrete-choice experiment (DCE) was designed to elicit public preferences for a new targeted approach to prescribing biologics ("biologic calculator") for rheumatoid arthritis (RA) compared with conventional prescribing. The DCE presented three alternatives, two biologic calculators and a conventional approach (opt out), described by five attributes: delay to treatment, positive predictive value, negative predictive value, infection risk, and cost saving to the national health service. Respondents were randomized to receive training materials as plain text or an animated storyline. Training materials contained information about RA and approaches to treatment and described the biologic calculator. Background questions included sociodemographics and self-reported measures of task difficulty and attribute non-attendance. DCE data were analyzed using conditional and heteroskedastic conditional logit (HCL) models. Results In total, 300 respondents completed the DCE, receiving either plain text (n = 158) or the animated storyline (n = 142). The HCL showed the estimated coefficients for all attributes aligned with a priori expectations and were statistically significant. The scale term was statistically significant, indicating that respondents who received plain-text materials had more random choices. Further tests suggested preference homogeneity after accounting for differences in scale. Conclusions Using animated training materials did not change the preferences of respondents, but they appeared to improve choice consistency, potentially allowing researchers to include more complex designs with increased numbers of attributes, levels, alternatives or choice sets.

Patient Preferences to Assess Value IN Gene Therapies: Protocol Development for the PAVING Study in Hemophilia

“…Comprehension of the survey by participants will be ensured through use of the educational tool that was designed. Vass et al ( 86 ) showed in their study that the use of an animated educational tool did not change preferences of respondents, but improved choice consistency. The information presented in the educational tool developed in the current research covers information needs of patients and was validated by hematologists and piloted with patients.…”

Patient preferences for gene therapy in haemophilia: Results from the PAVING threshold technique survey

“…31 Educational tools have also been found to play a crucial role in measuring consumer preferences for new products. 32,33 A significant effect of residence (mainly Flanders vs. Wallonia) on all benefit thresholds was observed. A correlation was observed between residence and prior discussion on gene therapy with a clinician as proportionally more PWH in Wallonia had discussed gene therapy with their clinician.…”

Patient preferences for stratified medicine in psoriasis: a discrete choice experiment*

“…The DCE survey went through an extensive piloting process (pilot survey with 82 patients; consultation with two academic dermatologists) that was run in parallel with a similar survey designed for people with RA 15 . Changes were made to the levels and their associated images for ‘cost saving to the National Health Service (NHS)’ based on the results from the quantitative pilot.…”

“…An iterative process, conducted alongside developing a similar survey for people with RA, identified the relevant attributes. 15 The results from interviews conducted as part of a qualitative study in RA 16 and five focus groups (attended by a total of 51 individuals with RA) were supplemented with a psoriasis support group meeting (seven individuals), literature review of psoriasis and DCEs, and two clinical expert interviews to inform the selection of attributes and to ensure that participants understood the survey. The psoriasis support group meeting involved collating views of the online survey by presenting and discussing the training materials and the framing of the attributes and levels.…”