Is there a credibility crisis in strategic management research? Evidence on the reproducibility of study findings

Abstract

Recent studies report an inability to replicate previously published research, leading some to suggest that scientific knowledge is facing a credibility crisis. In this essay, we provide evidence on whether strategic management research may itself be vulnerable to these concerns. We conducted a study whereby we attempted to reproduce the empirical findings of 88 articles appearing in the Strategic Management Journal using data reported in the articles themselves. About 70% of the studies did not disclose enough data to permit independent tests of reproducibility of their findings. Of those that could be retested, almost one-third reported hypotheses as statistically significant which were no longer so and far more significant results were found to be non-significant in the reproductions than in the opposite direction. Collectively, incomplete reporting practices, disclosure errors, and possible opportunism limit the reproducibility of most studies. Until disclosure standards and requirements change to include more complete reporting and facilitate tests of reproducibility, the strategic management field appears vulnerable to a credibility crisis.

An Empirical Assessment of the Influence of March and Simon’s Organizations: The Realized Contribution and Unfulfilled Promise of a Masterpiece

“…This trend has led some to claim that what we really have in management is a proliferation of pseudotheories (Cucina and McDaniel, 2016). The highly publicized 'replication crisis' in psychology, medicine, economics, and strategy has shown that much of the social science that we thought we knew is either simply untrue, or at least much less reliable than many scholars had assumed (e.g., Bergh et al, 2017;Camerer et al, 2016;Open Science Collaboration, 2015). As Tsang and Kwan (1999), Hunter (2001), and many others have argued, we need replications of tests of important theoretical assertions in order to have even a modicum of confidence that those assertions are true, and, to date, such replications have been rare.…”

Objectivity and Truth: The Role of the Essay in Management Scholarship

“…I was reminded of the questioner's challenge to the legitimacy of the essay recently when reading a series of papers that describe a growing legitimacy crisis in management scholarship. The first was a paper that tried and failed to reproduce the findings of a number of studies in a prominent strategic management journal (Bergh et al, ). The second was a paper, currently working its way through the review process, that demonstrates serious issues of ‘p‐hacking’ in elite management journals.…”

“…Like unfamiliar cultural standards of beauty such as foot binding in China and or ear stretching of the Maasai, the fundamental codes of a community often generates its own peculiar logics of worth.I was reminded of the questioner's challenge to the legitimacy of the essay recently when reading a series of papers that describe a growing legitimacy crisis in management scholarship. The first was a paper that tried and failed to reproduce the findings of a number of studies in a prominent strategic management journal (Bergh et al, 2017).…”

Endogeneity and Entrepreneurship Research

“…This opportunity-driven spirit infuses vigor into our scholarship, which is critically important to the development of the field. Yet as with our sister disciplines, vigor and being interesting are of limited use without accuracy; that is, vigor without rigor is vapid (Bergh et al, 2017). We have the opportunity to continue the remarkable evolution of entrepreneurship scholarship-and the value we provide to our stakeholders-by continuing to infuse rigorous methods with the fundamentally interesting questions we ask.…”

“…As the replication crisis in science grows, fields from medicine (Ioannidis, 2016), to psychology (Lindsay, 2017), to management (Bergh et al, 2017) are looking inward and acknowledging the challenges inherent to identifying, and reporting, accurate empirical findings. The desire to have each published paper make a novel-and ideally important-contribution to a theoretical conversation creates a perverse incentive to find results that are first and foremost 'interesting' (Butler et al, 2017;Goldfarb and King, 2016).…”