Explaining political participation: A comparison of real and falsified survey data

Abstract

Abstract. This paper examines differences between real survey data and data falsified by interviewers. Previous studies show that there are only small differences between real and falsified data which implies that falsifying interviewers are able to (re-)produce realistic frequency distributions. The question this paper aims to answer is whether they are also able to produce multivariate results in accordance with the assumptions of established social science approaches. As an example for a realistic theory-driven data analysis, real and falsified data are compared in terms of the identified determinants of political participation. I use an experimental data set with data partly collected in real interviews and partly by interviewers being instructed to falsify; that is, to fill in the questionnaire based on little information about the respondent. The questionnaire measures twelve political activities, based on which I calculate an index for political participation. There are differences in the models between the real and the falsified data: The explained variances are higher in the regression models of the falsified data. There are some variables significant in both data sets and some that are significant only in the real or in the falsified data. These differences can be explained by our theoretical assumptions.

Investigating Interviewer Falsifications – A Quasi-experimental Design

“…In the case of explaining political participation, we can find no differences between real and falsified data in that respect. Nevertheless we can conclude that the estimation of complex theory-driven models makes differences regarding the covariance structure between real and falsified data visible (Landrock and Menold, 2016; Landrock, 2017a).…”

“…To asses differences regarding the covariance structure, we developed the approach of estimating theory-driven models; this way we were able to research whether falsifiers are also able to construct complex models of behavior in accordance with social reality (Landrock and Menold, 2016; Landrock, 2017a). We estimated theory-driven OLS regressions and compared the results of the falsified with those of the real data.…”

“…Secondly, we adopted a rational choice model of explaining political participation (cp. Kaase and Marsh, 1979; Opp and Finkel, 2001) and, again, compared the results of falsified and of real data (Landrock, 2017a).…”

“… + positive significant effect (p < .05); - negative significant effect (p < .05); (+) positive effect p < .10; (-) negative effect (p < .10); (n.s.) not significant Source: Landrock, 2017a: Online preprint (condensed table) …”

How to Catch a Falsifier

“…Interviewer falsification may take various forms, such as intentional miscoding of respondents’ eligibility status or answers, deviations from instructions, and, the most severe form, the fabrication of complete interviews ( AAPOR 2003 ). Although empirical evidence suggests that complete falsification is a rare event ( Blasius and Friedrichs 2012 ), even small amounts of undetected fraudulent data can severely bias survey estimates, particularly in multivariate analyses ( Schräpler and Wagner 2005 ; Landrock 2017 ; DeMatteis et al 2020 ).…”