No abstract

No abstract

No abstract

Researchers make hundreds of decisions about data collection, preparation, and analysis in their research. We use a many-analysts approach to measure the extent and impact of these decisions. Two published causal empirical results are replicated by seven replicators each. We find large differences in data preparation and analysis decisions, many of which would not likely be reported in a publication. No two replicators reported the same sample size.Statistical significance varied across replications, and for one of the studies the effect's sign varied as well. The standard deviation of estimates across replications was 3-4 times the mean reported standard error.

No abstract

No abstract

Many researchers use an ordinal scale to quantitatively measure and analyze concepts. Theoretically valid empirical estimates are robust in sign to any monotonic increasing transformation of the ordinal scale. This presents challenges for the point-identification of important parameters of interest. I develop a partial identification method for testing the robustness of empirical estimates to a range of plausible monotonic increasing transformations of the ordinal scale. This method allows for the calculation of plausible bounds around effect estimates. I illustrate this method by revisiting analysis by Nunn and Wantchekon (2011, American Economic Review, 101, 3221–3252) on the slave trade and trust in sub-Saharan Africa. Supplemental illustrations examine results from (i) Aghion et al. (2016, American Economic Review, 106, 3869–3897) on creative destruction and subjective well-being and (ii) Bond and Lang (2013, The Review of Economics and Statistics, 95, 1468–1479) on the fragility of the black–white test score gap.

The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.