Survey participation effects in conflict research

Abstract

Do survey participants in conflict zones respond differently if they have been interviewed before? Academic and policy interest in postwar political opinion has increased tremendously. One unexpected consequence of this surge of survey research is a growing probability that individuals will be interviewed multiple times. However, if participating in one survey causes respondents to change their attitudes or behavior, their subsequent survey responses may be biased in comparison to the rest of the sample population. Our article aims to investigate such ‘survey participation effects’ in conflict contexts. We draw on original survey data collected in the eastern Democratic Republic of Congo (DRC). In our representative sample, 18% of respondents report that they have been interviewed before. Multivariate analyses demonstrate that their stated attitudes on social relations, political institutions, gender norms, and wartime victimization differ substantively from the responses of first-time interviewees. Moreover, our analyses indicate that experienced respondents have specific response styles – in particular, a tendency to support extreme response options. While substantive bias in multivariate analyses seems to be rather rare, our findings indicate that researchers should be aware of the footprints of data collection efforts in areas frequently targeted by household and opinion surveys.

Wartime Sexual Violence, Social Stigmatization and Humanitarian Aid: Survey Evidence from eastern Democratic Republic of Congo

“…Finally, to begin accounting for variables that may determine both reporting of and exposure to rape by armed groups that may simultaneously be determinants of stigma, we include controls for age , sex ( female ), and education of the respondent, household size and economic status ( assets ) in our basic models. We also control for respondents’ prior survey participation experience since this has been shown to affect response patterns (De Juan and Koos 2021). 12 Summary statistics are available in Supplemental Table A1 and the survey questions are described in section A4 in the appendix.…”

Social Preferences: Measuring Private, Public, and Group Preferences through Focus Groups

“…Literature on research ethics highlights the importance of considering “research fatigue,” the effect that repeated questioning about traumatizing past events has on respondents (Boesten and Henry 2018). Quantitative work has also revealed an effect of having been a research participant (in any form of research) on subsequent survey responses in this same context (De Juan and Koos 2021). If private preferences change as a result of research engagements, then there needs to be increased oversight and efforts to mitigate any potential harmful effects.…”

Claims to ignorance as a form of participation in transitional justice

“…Scholars ignore or under-theorise these responses. For example, De Juan and Koos (2021), in their analysis of survey participation effects in conflict research, hypothesise that individuals who have been surveyed more than once should have lower 'don't know' response rates, but they find no significant change to these rates. Gibson (2005) notes the lack of opinion on numerous questions in his research on the South African transitional justice process.…”

“…High rates of ‘don’t know’ responses are prevalent across much of transitional justice research. Surveys consistently find that particular questions or particular groups of respondents give a high proportion of ‘don’t know’ or ‘no response’ answers, even when it seems reasonable that everyone may have an opinion on the topic (see, for example, Aguilar et al, 2011; De Juan and Koos, 2021; Gibson, 2005; UNDP, 2012). A series of surveys conducted by IPSOS in Bosnia-Herzegovina (IPSOS Strategic Marketing/Belgrade Human Rights Centre, 2012a), Croatia (IPSOS Strategic Marketing/Belgrade Human Rights Centre, 2012b) and Serbia (IPSOS Strategic Marketing/Belgrade Human Rights Centre, 2009), which are also replicated in this study, illustrate this (Table 1).…”