This study explores how researchers’ analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers’ expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team’s workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers’ results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.
This article examines the links between education, democratic values, and political trust. Research on education systems as carriers of modern orientations and democratic values worldwide predicts that educated individuals will exhibit more democratic values than less educated ones, regardless of the country’s level of democracy. In the political culture approach, political trust can be understood as a reflection of the congruence or incongruence between individual democratic values and the level of democracy of the political system, which emphasizes trust’s relational character. Integrating these strands of literature, I formulate hypotheses about the mediating effect of democratic values between education and political trust. To test these hypotheses, I employ multilevel models of data from the World Values Survey and the European Values Study covering 73 countries. The results show that democratic values partially mediate the effect of education on political trust, but the magnitude of this effect depends on the level of democracy. Analyses also show that, while education is positively associated with democratic values regardless of the country’s level of democracy, this association is much stronger in democratic countries than in non-democratic ones.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.