Good survey and experimental research requires subjects to pay attention to questions and treatments, but many subjects do not. In this article, we discuss "Screeners" as a potential solution to this problem. We first demonstrate Screeners' power to reveal inattentive respondents and reduce noise. We then examine important but understudied questions about Screeners. We show that using a single Screener is not the most effective way to improve data quality. Instead, we recommend using multiple items to measure attention. We also show that Screener passage correlates with politically relevant characteristics, which limits the generalizability of studies that exclude failers. We conclude that attention is best measured using multiple Screener questions and that studies using Screeners can balance the goals of internal and external validity by presenting results conditional on different levels of attention. G ood survey and experimental research requires subjects to pay attention to questions and treatments, but not all people pay close attention all of the time. When respondents do not read questions carefully, their responses on related survey items can appear to be unrelated; when subjects do not pay attention to experimental treatments, replications of classic experiments can produce null results. As self-administered surveysboth online and in the lab-continue to grow in popularity, problems arising from inattentive respondents will also grow. Researchers must consider how best to identify and handle inattentive respondents.Instructional Manipulation Checks (IMCs), or "Screeners," are a potential solution to this problem and are increasingly common in political science and psychology (Oppenheimer, Meyvis, and Davidenko 2009).
Scholars have consistently shown that social identities can influence political attitudes and behaviors; this paper explores the reverse relationship. Are partisan identities ever strong enough to influence involvement with a politically relevant social group? Looking at an identity that has become an increasingly strong predictor of partisanship and vote choice, religion, the paper develops and tests a theory that politics can influence a partisan's religiosity at a certain time in his or her life. An experiment and two panel studies show that when people are in the process of raising children-a time that encourages many to make decisions associated with their religious identities-their partisanship can influence these religious choices. The findings highlight partisanship's ability to influence key aspects of partisans' social identities and, ultimately, the religious makeup of the United States. * For comments, suggestions, and feedback, I thank
Inattentive respondents introduce noise into data sets, weakening correlations between items and increasing the likelihood of null findings. “Screeners” have been proposed as a way to identify inattentive respondents, but questions remain regarding their implementation. First, what is the optimal number of Screeners for identifying inattentive respondents? Second, what types of Screener questions best capture inattention? In this paper, we address both of these questions. Using item-response theory to aggregate individual Screeners we find that four Screeners are sufficient to identify inattentive respondents. Moreover, two grid and two multiple choice questions work well. Our findings have relevance for applied survey research in political science and other disciplines. Most importantly, our recommendations enable the standardization of Screeners on future surveys.
White evangelicals overwhelmingly supported Donald Trump in the 2016 election, producing extensive debate as to who evangelicals are, what it means to be an evangelical in the United States today, and whether the electoral results are surprising or not. This paper offers empirical clarity to this protracted discussion by asking and answering a series of questions related to Trump's victory in general and his support from white evangelicals in particular. In doing so, the analyses show that the term “evangelical” has not become a synonym for conservative politics and that white evangelical support for Trump would be higher if public opinion scholars used a belief-centered definition of evangelicalism rather than relying on the more common classification strategies based on self-identification or religious denomination. These findings go against claims that nominal evangelicals, those who call themselves evangelicals but are not religious, make up the core of Trump's support base. Moreover, strong electoral support among devout evangelicals is not unique to the 2016 election but rather is part of a broader trend of evangelical electoral behavior, even when faced with non-traditional Republican candidates. Finally, the paper explores why white evangelicals might support a candidate like Trump. The paper presents evidence that negative partisanship helps explain why devout evangelicals—despite Trump's background and behaviors being cause for concern—coalesced around his presidential bid. Together, the findings from this paper help make sense of both the 2016 presidential election and evangelical public opinion, both separately and together.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.