This study explores how researchers’ analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers’ expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team’s workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers’ results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.

No abstract

Pension adequacy is gaining importance as old-age poverty remains a pressing problem. In many advanced welfare states, the population is ageing rapidly and recent pension reforms have led to cuts in public pension provision. There are, however, few comparative longitudinal studies on the relationship between pension generosity and old-age poverty. This study provides a comparative empirical assessment of how the prevalence and depth of old-age poverty relates to generosity of public pension benefits in 14 advanced OECD welfare states from 1980-2010. We focus on the role of mandatory public pension provision of mainly first tier schemes that grant the major share of retirees’ income in most countries. We use data on theoretical pension replacement rates for retirees who had different working-age incomes. In order to address endogeneity issues, we adopt an instrumental-variable approach. Our main finding shows that pensions systems and earnings-related schemes, in particular, are quite efficient in reducing the risk of old-age poverty. Yet they still do very little to alleviate poverty among those pensioners in the most disadvantaged situations. We also found that redistribution within the pension system does not substantially contribute to poverty alleviation.

Background Previous research suggests that school exclusion during childhood is a precursor to social exclusion in adulthood. Past literature on the consequences of school exclusion is, however, scarce and mainly focused on short‐term outcomes such as educational attainment, delinquency, and mental health in early adolescence. Moreover, this evidence is based primarily on descriptive and correlational analysis, whereas robust causal evidence is required to best inform policy. Aims We aimed to estimate the mid‐to‐long‐term impact of school exclusion on labour market and economic outcomes. Sample The sample included 6,632 young people who at the age of 25/26 in the year 2015 participated in the Next Steps survey of whom 86 were expelled from school and 711 were suspended between the ages of 13/14 and 16/17. Method Using high quality existing longitudinal data, we utilized four approaches to evaluate the impact of school exclusion: logistic regression‐adjustment models, propensity score matching, school fixed‐effects analysis, and inverse propensity weighting. The latter two counterfactual approaches were used to estimate causal effects. Results We found that school exclusion increased the risk of becoming NEET at the age of 19/20, and then remaining economically inactive at the age of 25/26, as well as experiencing higher unemployment risk and earning lower wages also at the age of 25/26. Conclusion School exclusion has pervasive negative effects into adulthood. Policy interventions should focus on both prevention and mitigating its negative effects. Interventions aimed at re‐integrating excluded individuals into education or vocational training could be key in reducing the risk of poor socio‐economic outcomes and social exclusion.