Dawning of a new age? Economics journals’ data policies on the test bench

Abstract

In the field of social sciences and particularly in economics, studies have frequently reported a lack of reproducibility of published research. Most often, this is due to the unavailability of data reproducing the findings of a study. However, over the past years, debates on open science practices and reproducible research have become stronger and louder among research funders, learned societies, and research organisations. Many of these have started to implement data policies to overcome these shortcomings. Against this background, the article asks if there have been changes in the way economics journals handle data and other materials that are crucial to reproduce the findings of empirical articles. For this purpose, all journals listed in the Clarivate Analytics Journal Citation Reports edition for economics have been evaluated for policies on the disclosure of research data. The article describes the characteristics of these data policies and explicates their requirements. Moreover, it compares the current findings with the situation some years ago. The results show significant changes in the way journals handle data in the publication process. Research libraries can use the findings of this study for their advisory activities to best support researchers in submitting and providing data as required by journals.

Statistical reporting errors in economics

“…This finding places even more emphasis on the need for such policies in addition to their benefits in terms of replicability. Although these policies have markedly increased over time to nearly 20 % of all empirically-oriented economics journals having a mandatory data availability policy (Vlaeminck 2021), we suggest that all journals implement not only data availability policies but also code availability policies.…”

“…Note that Pütz and Bruns (2021) focused on the actual availability of data and/or code per published article while we focus on journal policies. 23 Note that, following the definition of Vlaeminck (2021), we focus on mandatory data availability policies which require authors to upload their replication files to a third party prior publication, while we neglect policies that only encourage authors to upload their files or author responsibility policies which leave the responsibility to share data to the authors. While Nuijten et al (2017) do not find any association between reporting errors and data availability policies in psychology, Wicherts et al (2011) show that the unwillingness to share data is associated with the prevalence of reporting errors.…”

Reproducibility in Management Science

“…In recent years, there have been significant developments in the institutional arrangements for reproducibility of journal articles. For economics, Vlaeminck (2021) The ability to reproduce results reported in published articles by executing the code on the data, both provided by the authors, does not, by itself, guarantee that results are replicable. But it does provide a useful baseline.…”

Políticas editoriais de compartilhamento de dados em periódicos brasileiros de ciências sociais aplicadas na coleção SciELO

“…Além disso, apenas um quarto dos periódicos com políticas de dados subscrevem os princípios FAIR. Esses resultados sugerem que, na amostra analisada, as políticas de compartilhamento de dados, quando existentes, são vagas e incompletas, corroborando os achados deVlaeminck (2021) e Jackson (2021) e Christian et al (2020) também encontraram políticas vagas, incompletas e mesmo contraditórias em periódicos de diversas áreas (inclusive nas Ciências Sociais) e observaram que a falta de informações claras e precisas pode levar autores e editores a criarem expectativas divergentes com relação ao compartilhamento de dados.…”