Dropping subjects based on the results of a manipulation check following treatment assignment is common practice across the social sciences, presumably to restrict estimates to a subpopulation of subjects who understand the experimental prompt. We show that this practice can lead to serious bias and argue for a focus on what is revealed without discarding subjects. Generalizing results developed in Zhang and Rubin (2003) and Lee (2009) to the case of multiple treatments, we provide sharp bounds for potential outcomes among those who would pass a manipulation check regardless of treatment assignment. These bounds may have large or infinite width, implying that this inferential target is often out of reach. As an application, we replicate Press, Sagan, and Valentino (2013) with a design that does not drop subjects that failed the manipulation check and show that the findings are likely stronger than originally reported. We conclude with suggestions for practice, namely alterations to the experimental design.
Local leaders possess significant and growing authority over refugee resettlement, yet we know little about their attitudes toward refugees. In this article, we use a conjoint experiment to evaluate how the attributes of hypothetical refugee groups influence local policymaker receptivity toward refugee resettlement. We sample from a national panel of current local elected officials, who represent a broad range of urban and rural communities across the United States. We find that many local officials favor refugee resettlement, regardless of refugee attributes. However, officials are most receptive to refugees whom they perceive as a strong economic and social fit within their communities. Our study contributes to a growing literature on individual attitudes toward refugees by systematically examining the preferences of US local elected officials and offers unique insights into the views of this influential and policy-relevant group.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.