Being surveyed can change later behavior and related parameter estimates

Abstract

Does completing a household survey change the later behavior of those surveyed? In three field studies of health and two of microlending, we randomly assigned subjects to be surveyed about health and/or household finances and then measured subsequent use of a related product with data that does not rely on subjects' selfreports. In the three health experiments, we find that being surveyed increases use of water treatment products and take-up of medical insurance. Frequent surveys on reported diarrhea also led to biased estimates of the impact of improved source water quality. In two microlending studies, we do not find an effect of being surveyed on borrowing behavior. The results suggest that limited attention could play an important but context-dependent role in consumer choice, with the implication that researchers should reconsider whether, how, and how much to survey their subjects. measurement effects | question-behavior effects | Hawthorne effects | survey methodology | models of attention M any data collection efforts in the social and clinical sciences rely on surveys. Psychologists and marketing and survey experts have long held that surveying a subject can draw attention to risks or choices with ordinarily little salience and thereby induce changes in subsequent behavior (1, 2), but conclusive evidence on this question from which a causal relationship can be clearly established has been limited. We provide evidence from a variety of settings that the act of being surveyed can affect behavior and confound estimates of parameters that initially motivated the data collection."Survey" or "interview" effects may occur even when the survey does not ask specifically about intent to engage in the behavior of interest or provide new information, and even when subjects do not know that their later behavior is being observed by researchers. Hence survey effects are conceptually distinct from, but closely related to, "question-behavior" (i.e., mere measurement or self-prophecy) and Hawthorne effects. Question-behavior effects arise when behavior changes as a result of asking subjects for intentions or predictions regarding future behavior effects (3-6). Hawthorne effects occur when behavior changes as a result of a subject responding to being treated and observed, as part of an experiment (7). These effects are also related to "push polling" and other efforts to manipulate subject behavior by posing hypothetical questions (8).We describe results from five different field experiments, in four developing countries, on whether being surveyed affects subsequent behavior. The first experiment randomizes the frequency of surveys on health behavior and diarrhea incidence, in a context in which other water quality interventions are also randomly provided. More-frequent surveying leads to lower reported child diarrhea and cleaner water (as measured by the presence of detectable chlorine in household drinking water). We posit that frequent surveying serves as a reminder to invest in water purification, although because sub...

A community randomised controlled trial evaluating a home-based environmental intervention package of improved stoves, solar water disinfection and kitchen sinks in rural Peru: Rationale, trial design and baseline findings

“…The use of LP and monthly outcome measurements is further constraint as the authors illustrate the need to considerably increase sample size maintaining the same power of the study [36]. We concur with Zwane and colleagues that frequent contacts and measurements may influence illness reporting and, thus, potentially bias intervention effects [42]. However, given the features of our interventions and household surveillance in both study arms we believe that any potentially resulting bias will be balanced.…”

Use, microbiological effectiveness and health impact of a household water filter intervention in rural Rwanda—A matched cohort study

“…Additionally, the implementers remained programmatically engaged with communities throughout the study. This could have influenced household behaviours and responses McCarney et al, 2007;Zwane et al, 2011). There was also high attrition in this study (12.6%), particularly among the intervention arm (19.5%).…”

The Effect of Water Quality Testing on Household Behavior: Evidence from an Experiment in Rural India

“…In particular, it is difficult to know whether declines in self-reported measures of protective behaviors across the entire sample were the result of seasonal adjustments or some other factors. 19 Our DiD approach is likely, however, to sweep out biases arising from misclassification in the self-reports. In addition, our study does not provide data on the extent to which behavior change led to measurable improvements in water quality, which others have shown to be more difficult.…”