Strategies for Dealing with the Problem of Non-overlapping Units of Assignment and Outcome Measurement in Field Experiments

Abstract

Researchers conducting field experiments are sometimes faced with the challenge of analyzing field experiment results when the unit of assignment does not coincide with the unit of outcome measurement. For example, in electoral research, election results may be reported at a level of geography defined by electoral law, while the assignment of treatment can be made only at a level of geography different from this. Using examples from field experiments conducted in Canada and Mexico, we describe this problem and its consequences for analysis and interpretation of field experiment data and results. We also offer a number of practical solutions analysts can employ when faced with non-overlapping units of assignment and outcome measure in field experiments.

Ancillary Studies of Experiments: Opportunities and Challenges

“…An alternative solution is to directly take into account the characteristics of the ancillary units that condition their probability of exposure to the treatment. Researchers can identify the effect of receiving treatment by stratifying ancillary units according to their probability of receiving treatment (De La O and Rubenson 2010). For example, De La O is able to identify the effect of PROGRESA on vote returns by separately analysing precincts with different numbers of experimental villages.…”