Do economists replicate?

Promoting Reproducibility and Replicability in Political Science

“…Several definitions of reproducibility and replicability have been used and proposed (see, e.g., Clemens 2017;Christensen and Miguel 2018;Ankel-Peters et al, 2023a). Dreber and Johannesson (2023) recently proposed definitions and indicators for economics that we summarize here and believe can be useful also for political science.…”

“…In economics, the Journal of Political Economy and Labor Economics discontinued formats dedicated to replication due to a lack of submissions (Hamermesh 2007). Likewise, the American Economic Review features an online format for replications with only one published comment, according to the editors due to a lack of submissions as well (see Ankel-Peters, 2023b). Hence, there are reasons to believe that the dearth of replications in journals is also a supply problem, probably because scholars have little incentives, intrinsic or extrinsic, to engage in reproductions and replications.…”

Forking paths in empirical studies

“…See also Shepperd and Yousefi (2023) for an analysis of retractions in computer science. 3 We point to Boylan (2016), Duvendack et al (2017), Christensen and Miguel (2018), Mueller-Langer et al (2019), Vilhuber (2020), Colliard et al (2022), Hofman et al (2021), Cortina et al (2022), Peters et al (2023) and Vu (2022) for discussions on replication and reasons that explain why it may be hard. Recent initiatives are the Institute for Replication in economics and the Open Marketing platform.…”

Replications in agricultural economics

“…In both instances, the fundamental role of science could be undermined. Considering this, replication of published work can be an important vehicle to verify results, that is, uncover errors and fraud, as well as to test the generalizability of previous findings in new contexts (Fiala et al, 2022).…”

“…Moreover, effect sizes were found to be often smaller in replication studies, for instance, by Brodeur et al (2016Brodeur et al ( , 2020, Camerer et al (2016), Christensen and Miguel (2018), Ferraro and Shukla (2020), Huntington-Klein et al (2021), Ioannidis et al (2017), Peters et al (2018), and Vivalt (2020). An overview is provided by Fiala et al (2022).…”