Measuring Political Knowledge in Online Surveys

Abstract

Individuals use the internet to look up information. This has consequences for the measurement of political knowledge in self-administered online surveys. Online surveys may now include answers that were retrieved from the internet instead of a respondent’s declarative memory, thereby distorting knowledge measures. This problem has been acknowledged and studied, but existing research focuses on interventions designed to deter online searching and on methods to detect the behavior. I take a different approach and focus on the questions themselves to show that some questions are more vulnerable to bias via internet searching than others. I take advantage of interview mode differences in the 2016 ANES to examine the effect of internet access on different kinds of political knowledge questions. The study concludes with best practices recommendations for researchers wishing to improve their knowledge scales by discussing what type of questions are most likely to form a scale that retains criterion validity in the online environment.