Trust in Survey Results. A Cross-country Replication Experiment

Abstract

Amidst a flood of new survey results, people selectively determine which results to trust. These decisions may be shaped by perceived methodological quality, the survey’s sponsor or platform, and various individual political and non-political characteristics. This study examined the factors that drive trustworthiness assessments of survey results in the two countries. Study 1 replicated the study of Stadtmüller et al. (2022), Study 2 extended the original study by applying the same experimental design to a more contested and politicized issue (immigration). In online panel surveys, respondents were asked to rate hypothetical survey results. Survey quality information, sponsors, and survey results varied in the vignettes. The results show that most of the findings of Stadtmüller et al. (2022) travel well across survey topics and countries. Individual characteristics explain more than survey quality information, larger sample sizes enhance trust, and individuals with higher level of education attribute more importance to methodological information, regardless of the topic, in both countries. Our study brought nuance to the original findings by showing that the extent to which individuals incorporate survey quality information depends on the degree of controversy and politicization associated with the survey topic.