Respondent Conditioning in Online Panel Surveys

Abstract

In this article, we investigate changes in survey reporting due to prior interviewing. Two field experiments were implemented in a probability-based online panel in which the order of the questionnaires was switched. Although experimental methods for studying panel conditioning are favorable, experiments in longitudinal studies are rare. Studies on conditioning demand additional resources and might influence respondents' answers. Panel conditioning is mostly associated with measurement errors. However, the discussion that sees it exclusively as a negative phenomenon is not comprehensive. Learning the rules of the interview may lead to increases or decreases in data quality (advantageous vs. disadvantageous conditioning). Overall, little evidence of advantageous conditioning and no disadvantageous conditioning is found. Apart from this reassuring finding, this aricle advances the field by using propensity score weighting to account for attrition and other confounding factors and by using paradata to evaluate the plausibility of alternative explanations of panel conditioning.

What was I thinking? A theoretical framework for analysing panel conditioning in attitudes and (response) behaviour

“…In the absence of external factors, the avoidance of lengthy questions is clearly attributable to a desire to reduce burden and can thus be regarded as a problem of response behaviour (Struminskaya, 2016). However, more exact reporting has been proposed -sometimes even simultaneously -as a second memory effect (Silberstein & Jacobs, 1989;Waterton & Lievesley, 1989).…”

“…Instead, we give an overview on popular streams of proposed theoretical explanations for panel conditioning effects in the realm of attitudes and (response) behaviour. It has repeatedly been remarked that the theoretical basis of research concerning panel conditioning effects is rather thin (Struminskaya, 2016;Sturgis et al, 2009;Warren & Halpern-Manners, 2012). While a number of sound assumptions have been formulated, most propositions are not linked to a broader theoretical framework.…”

“…In this respect, learning the rules that govern a survey interview might also lead to better informed respondents who give more exact answers. Additionally, familiarization with the survey procedure due to repeated participation is supposed to increase respondents' trust, which is why more truthful answers are expected in later waves (Struminskaya, 2016;Waterton & Lievesley, 1989). Finally, respondents remembering their answers from earlier waves may be disinclined to change their once stated opinion as they are reluctant to appear fickle, resulting in higher response consistency (Crespi, 1948).…”

“…While a substantial number of studies has suggested sophisticated measurement approaches for the separation of panel conditioning effects from attrition bias and true change (Crossley, de Bresser, Delaney, & Winter, 2017;Das, Toepoel, & van Soest, 2011;Struminskaya, 2016;Yan & Eckman, 2012), the phenomenon's underlying mechanisms and driving factors are still far from clear. Hence, this paper aims to integrate proposed mechanisms behind panel conditioning in a comprehensive theoretical framework by drawing on insights from research on cognitive information processing, attitude strength and respondents' survey response strategies.…”

“…Crespi, 1948;Struminskaya, 2016;Waterton & Lievesley, 1989) or postulating some kind of stimulation by answering survey questions (e.g. Jagodzinski, Kühnel, & Schmidt, 1987;Sturgis, Allum, & Brunton-Smith, 2009).…”