Publication Bias in Reports of Animal Stroke Studies Leads to Major Overstatement of Efficacy

Abstract

Publication bias confounds attempts to use systematic reviews to assess the efficacy of various interventions tested in experiments modelling acute ischaemic stroke, leading to a 30% overstatement of efficacy of interventions tested in animals.

NIA Commentary: Translational Issues in Alzheimer’s Disease Drug Development

“…This has led to systematic bias, inadequate collection of data, and incorrect conclusions about animal efficacy. Animal studies must avoid bias by using power analysis, animal randomization, double blinding, and intention-to-treat analysis in the design of preclinical efficacy studies (van der Worp et al, 2010).…”

A systematic review and meta‐analysis of gene therapy in animal models of cerebral glioma: why did promise not translate to human therapy?

“…In addition, stratified meta‐analysis can explore the impact of independent study design variables (termed external validity) on reported outcome; assess the prevalence and impact of measures to reduce bias such as randomization, blinded outcome assessment and sample size calculations (termed internal validity); and can provide evidence of possible publication bias—a phenomenon where comparative over‐reporting of small efficacious studies versus small ineffective studies leads to a false overestimation of the benefit of a given therapy 25, 26. Several previous studies on experimental models of neurological disease have demonstrated the flaws in the internal validity of studies and have shown that reporting of such measures can significantly affect efficacy estimates 15, 27, 28, 29. Consequently, assessing these features forms a critical domain of the systematic review and meta‐analysis in preclinical literature.…”

Introducing Therioepistemology: the study of how knowledge is gained from animal research

“…There are patterns and principles that can help us identify models and results that are more or less likely to translate, and there are also easily realized, simple changes in the execution of animal work that will inherently improve translation [16][17][18] . This isn't a new concept; looking back over the last 10-15 years we can see many authors have been candid about the merits, strengths, weaknesses, reproducibility, and translatability of various animal models 1,2,[4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19] . Our goal with this article is to unite the common themes in this broader emerging literature and this special issue.…”

“…Indeed pharmaceutical companies continue to disinvest in internal animal R&D, a trend begun in the last decade, passing on the cost and risk to academia and startups 5,20 . Even this approach is not foolproof as pharmaceutical companies often cannot replicate the results of published work from academia 5,8,13 . Accordingly, there is a growing trend to focus on human, not animal, work for basic discovery 17 .…”

“…17,18,26-28,39,41) do not represent a voice in the wilderness, but one voice in a chorus and that this emerging literature 1,2,[4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40]42,43 reflects a nascent discipline which can be codified as the study of how knowledge is gained from animal research. We propose the title "Therioepistemology" This focus issue of Lab Animal coincides with a tipping point in biomedical research.…”