Reproducibility in Accounting Research: Views of the Research Community

Abstract

We have little knowledge about the prevalence of irreproducibility in the accounting literature. To narrow this gap, we conducted a survey among the participants of the 2019 JAR Conference on their perceptions of the frequency, causes, and consequences of irreproducible research published in accounting journals. A majority of respondents believe that irreproducibility is common in the literature, constitutes a major problem, and receives too little attention. Most have encountered irreproducibility in the work of others (although not in their own work) but chose not to pursue their failed reproduction attempts to publication. Respondents believe irreproducibility results chiefly from career or publication incentives as well as from selective reporting of results. They also believe that practices like sharing code and data combined with stronger incentives to replicate the work of others would enhance reproducibility. The views of accounting researchers are remarkably similar to those expressed in a survey by the scientific journal Nature. We conclude by discussing the implications of our findings and provide several potential paths forward for the accounting research community.

Understanding How the Effects of Conditional Conservatism Measurement Bias Vary with the Research Context

“…Given the importance of CC as a research topic in the accounting literature, and the ongoing debate over the validity of the widely adopted AT measure of Basu (1997), we believe that it is necessary to heed the calls of Ball (2016) and Hail et al (2020) to revisit inferences drawn from previous studies using more recently developed research methodologies. As such, unlike previous studies involved in the debate over the validity of the AT measure, we contribute to the CC literature by providing direct evidence on whether and how bias in the AT measure varies across frequently adopted research settings.…”

“…In response to this controversy, we heed the call of Ball (2016b) and Hail, Lang, & Leuz (2020) to revisit previous empirical studies and verify their conclusions using new methodologies and measures.…”

Measurement Operationalization and Written Clarity in Accounting Research: An Integrated Pedagogical Approach for a Doctoral Research Seminar

“…Replications of these studies using appropriate statistical methods and time periods may lead to different results. Hail et al (2020) performed a survey among 2019 Journal of Accounting Research Conference partakers regarding their views of the causes, effects, and frequency of irreplicable research in accounting journals. Results show that sharing code and data along with greater incentives to reproduce others' work would improve replicability.…”

Is One Head Better Than Two? Dual Leadership and Firm Performance During the COVID‐19 Crisis

“…By testing and confirming the validity of Yang and Zhao's finding in a different research setting, we also contribute to the reproducibility literature. As Hail et al (2020) point out, checking the authenticity of prior research findings in similar contexts or alternative periods enhances the credibility of the results and the entire field.…”

“…Thus, an exogenous crisis offers a better setting than import tariff rate reduction and economic policy uncertainty to test the efficacy of CEO duality and underlying decisions. In addition, testing the validity of a previous research finding in another setting or period (i.e., reproducibility) enhances the credibility of their results, carries practical relevance to policymaking and regulation (Hail et al, 2020), and helps generalize a finding across populations (Cready, 2021).…”