Making peer review evidence-based: It’s time to open the "black box"

Abstract

Peer review serves an essential role in the cultivation, validation, and dissemination of social work knowledge and scholarship. Nevertheless, the current peer review system has many limitations. It is charged as being unreliable, biased, ineffective, and unaccountable, among numerous other issues. That said, peer review is still commonly viewed as the best possible system of knowledge governance, given the relevant alternatives. In this research note, I scrutinize this assumption. Although peer review can sometimes be effective, it is not therefore a rigorous or even dependable system. Indeed, the practice of peer review in social work is overwhelmingly closed and opaque, and assurances of its rigor are speculative at best. Given that social work research informs policies and practices that have real world consequences for clients and communities, it is imperative that our research – and its appraisal – be held to the strictest of standards. This includes our system(s) of peer review. After highlighting common criticisms of traditional peer review, I articulate a research agenda on “open peer review” which can reform how peer review is performed, provide feedback to editors and reviewers, and help make the process more rigorous, transparent, and evidence-based. Implications for social work education are explored and discussed.