Authorization of Animal Experiments Is Based on Confidence Rather than Evidence of Scientific Rigor

Abstract

Accumulating evidence indicates high risk of bias in preclinical animal research, questioning the scientific validity and reproducibility of published research findings. Systematic reviews found low rates of reporting of measures against risks of bias in the published literature (e.g., randomization, blinding, sample size calculation) and a correlation between low reporting rates and inflated treatment effects. That most animal research undergoes peer review or ethical review would offer the possibility to detect risks of bias at an earlier stage, before the research has been conducted. For example, in Switzerland, animal experiments are licensed based on a detailed description of the study protocol and a harm–benefit analysis. We therefore screened applications for animal experiments submitted to Swiss authorities (n = 1,277) for the rates at which the use of seven basic measures against bias (allocation concealment, blinding, randomization, sample size calculation, inclusion/exclusion criteria, primary outcome variable, and statistical analysis plan) were described and compared them with the reporting rates of the same measures in a representative sub-sample of publications (n = 50) resulting from studies described in these applications. Measures against bias were described at very low rates, ranging on average from 2.4% for statistical analysis plan to 19% for primary outcome variable in applications for animal experiments, and from 0.0% for sample size calculation to 34% for statistical analysis plan in publications from these experiments. Calculating an internal validity score (IVS) based on the proportion of the seven measures against bias, we found a weak positive correlation between the IVS of applications and that of publications (Spearman’s rho = 0.34, p = 0.014), indicating that the rates of description of these measures in applications partly predict their rates of reporting in publications. These results indicate that the authorities licensing animal experiments are lacking important information about experimental conduct that determines the scientific validity of the findings, which may be critical for the weight attributed to the benefit of the research in the harm–benefit analysis. Similar to manuscripts getting accepted for publication despite poor reporting of measures against bias, applications for animal experiments may often be approved based on implicit confidence rather than explicit evidence of scientific rigor. Our findings shed serious doubt on the current authorization procedure for animal experiments, as well as the peer-review process for scientific publications, which in the long run may undermine the credibility of research. Developing existing authorization procedures that are already in place in many countries towards a preregistration system for animal research is one promising way to reform the system. This would not only benefit the scientific validity of findings from animal experiments but also help to avoid unnecessary harm to animals for inconclusive research.

Sex bias in preclinical research and an exploration of how to change the status quo

“…This, however, is a huge change in perspective for the pipeline that manages in vivo research and not only includes the individual scientists but also the government review bodies, such as the UK Home Office, and institutional ethical review bodies. To date, ethical review bodies have focused on minimizing harm and typically assume the validity and replicability of the studies are met (Vogt et al ., ). Within an institute, the ethical review body can be a powerful component and can shape the designs run within an institute and the culture around the default position on sex.…”

A randomised controlled trial of an Intervention to Improve Compliance with the ARRIVE guidelines (IICARus)

“…We need therefore a better understanding of the barriers to implementing quality checklists for animal experiments. It has been suggested that requesting checklist adherence at the submission stage may be too late, given the observed correlation between reporting at the planning application stage and at the publication stage (Vogt et al, 2016). The PREPARE (Planning Research and Experimental Procedures on Animals: Recommendations for Excellence) guidelines (Smith et al, 2018) were published recently and may be a useful tool, in combination with the ARRIVE checklist, to promote a greater focus on experimental rigour at all stages of the research cycle.…”

Pre-Clinical Testing of Therapies for Traumatic Brain Injury

“…Corrigan and colleagues 67 observed that follow-up bias can occur in patient populations with lower socioeconomic status or in those patients who sustained a TBI due to violence or had a history of drug abuse. Additionally, based on a survey of 24 meta-analyses of RCTs, Page and co-workers 68 observed that the effects of interventions may be exaggerated by the 158 ; Fanelli et al 159 ; Vogt et al 160 ; Tsilidis et al 161 ; O'Collins et al 162 Modified from: Hawryluk et al 15 ; Margulies and Hicks 121 . DAI, diffuse axonal injury; RCT, randomized clinical trial.…”